Selection (General) Flashcards

Question

Job simulations for selection

Answer 1

- applicants demonstrate actual job behaviors | - Highly content valid, but impractical

Answer 2

- work sample tests | - assessment centers

Answer 3

Pros - High content validity: directly relate to job tasks - Have great criterion validity - Great face validity, challenged less often in court - Lower racial differences in test scores than written cognitive ability tests Cons -Expensive to construct and administer

Answer 4

-Use multiple assessment methods and multiple assessors to actually observe applicants perform simulated job tasks Pros Job related methods and multiple trained assessors help protect against many types of selection bias

Answer 5

- Develop exercises that measure different aspects of the job - In baskets, modest support for usefulness - Simulations, lower adverse impact than traditional paper pencil tests - Leaderless group discussions - Business games

Answer 6

- Good at predicting performance, but there are cheaper methods to achieving the same or better predictive results - Adverse impact relatively high

Answer 7

- Experience ratings | - biodata

Answer 8

- Considers applicant’s life, school, military, community, and work experience - Good predictor of job performance, and best predictor of future employee tenure - Instrument is usually application blank or questionnaire, and these are weighted

Answer 9

- Predict work behaviors well in many jobs - Predict various performance criteria such as supervisor ratings, absenteeism, accidents, theft, tenure - Easy to use, quick to administer, inexpensive, - not subject to biases as other methods

Answer 10

1. Obtain employee information File approach Obtaining from personnel files on employees’ previous employment, education, etc. But information is often missing or incomplete Questionnaire approach Information cannot be obtained on employees who quit or were fired 2. Choose criterion 3. Criterion groups Employees are split into two groups based on their criterion scores 4. Compare employee information Each piece of employee information is compared with criterion group membership to determine which pieces of information will distinguish members of high group from low group

Answer 11

Validity may not be stable; ability to predict performance may decrease over time Some biodata items may not meet legal requirements in Uniform Guidelines Can be faked

Answer 12

predict performance well and have less adverse impact than ability tests

Answer 13

The best interpretation across meta analyses is: - Personality can predict performance at low but significant levels - Can add incremental validity to use of other selection tests - Conscientiousness is best predictor across most jobs and performance criterion - Validity of the other 4 traits is dependent on type of job and criterion

Answer 14

- Self reports, easy to fake, BUT research indicates that applicants fake less often than thought, and when they do, it doesn’t really effect the validity of the test results - Validity of personality inventories asking about work personality is more valid than asking about personality in general

Answer 15

Unstructured interviews Education Interest inventories Some personality traits

Answer 16

Ability Work samples Biodata Structured interviews

Answer 17

Cognitive ability test, and either a work sample, integrity test, or structured interview ; A well constructed selection battery will use a variety of testing methods in order to tap into various dimensions of the job

Answer 18

Interviews, work samples, resumes are perceived as most job related/fair

Answer 19

Graphology, integrity tests, and personality tests perceived as least job related/fair

Answer 20

it's super valid, and increases in validity as the Job gets more complex (i.e., its relationship to job performance increases as the job gets more complex; Schmidtt & Hunter 1988) it can be used for basically all jobs, and it's cheap to administer. HOWEVER it often results in adverse impact and has the largest white black subgroup difference so many orgs are moving away from it, and towards things like including content valid job specific ability tests that have way lower group differences , GMA is also less likely to be viewed as fair by candidates.

Answer 21

Judges estimate the probability that a minimally qualified candidate answers the item correctly, or the percent of minimally qualified candidates who would correctly respond to the item A prediction of item difficulty is the product of this data. Item difficulties are summed and a predicted test score is obtained for each judge. Then the mean of these predicted test scores from all judges are averaged and used as the cut off score.

Answer 22

Criterion method: setting a cut score is independent of test results. Used to judge performance against a set benchmark. Usually involves SME input, which is labor intensive, costly, and subjective. Norm-referenced: cut scores are set depending on test results, and norms are made. This is usually used to rank people

Answer 23

Restricts the probability choices for SMEs to 8 choices. The purpose is to apply the Angoff method but without overburdening SMEs. Subject matter experts are generally briefed on the Angoff method and allowed to take the test with the performance levels in mind. SMEs are then asked to provide estimates for each question of the proportion of borderline or “minimally acceptable” participants that they would expect to get the question correct. The estimates are generally in p-value type form (e.g., 0.6 for item 1: 60% of borderline passing participants would get this question correct). Several rounds are generally conducted with SMEs allowed to modify their estimates given different types of information (e.g., actual participant performance information on each question, other SME estimates, etc.). The final determination of the cut score is then made (e.g., by averaging estimates or taking the median). This method is generally used with multiple-choice questions.

Answer 24

SMEs make decisions on a question-by-question basis regarding which of the question distracters they feel borderline participants would be able to eliminate as incorrect. This method is generally used with multiple-choice questions only.

Answer 25

local norms are developed that help setting cut off scores. there are three main methods thordikes predicted yield: use information to determine score based on percentage of applicants that need to be hired to fill a position expectancy chart method: use expected percentage to identify the score associated with that percentile minus one SE. using job incumbents: give them the measure and then use that as the basis for deriving a cut off score. whatever you use make sure its justifiable legally in that it represents a useful and meaningful selection standard with regards to job performance

Answer 26

SMEs judge content of selection procedures and that helps determine the cut off score. mostly used for multiple choice or written tests. Ebel method: uses item difficulty where SMEs judge what percentage of the items a borderline taker would be able to answer correctly Angoff and modified Angoff discussed in other flash cards

Answer 27

at least 10-20% of the sample of SMEs representative of race gender shift etc. of the employe group; this means that experience of SMEs on the job under study is critical.

Answer 28

used to determine which parts of an application determine successful vs. unsuccessful performance by scoring and weighting individual application items use in these situations: jobs in which a large number of employees perform similar activities jobs in which good personnel records are available jobs that require long and costly training programs jobs with high turnover rate jobs with large number of applicants relative to position openings jobs in which it is expensive to bring applicants into the organization for interviewing and testing usually used for lower skilled jobs how to develop: 1. choose the criterion: could be job tenure, training program success, job performance. it's preferred this data be behavioral measures to increase reliability. 2. identify criterion groups: two groups of high vs. low criterion employees are distinguished. you want as many people as you can get for this step 3. selecting application blank items: depend on content and number of items on application form itself which should be derived from JA information 4. specify item response categories: developed for each WAB item to serve as a way of scoring applicant responses 5. determine item weights: based on degree of relationship of the WAB item with the criterion of success 6. apply weights to holdout groups: can't use the same group to determine success of the WAB so must cross validate. can use new applicants to do this. must wait for them to be at the org for considerable amount of time though 7. evaluate holdout group's WAB scores: plot their total WAB scores; high success employees should have higher WAB scores than lower success employees 8. set cut off scores to use for selection: maximize differentiation between successful vs. unsuccessful keep in mind that a statistical relationship between the WAB and success must also practically relate to the job in order to withstand legal scrutiny; just because it's statistically significant doesn't mean it should actually be used.

Answer 29

concerns personal background and life experiences presented in self report standardizes MC format usually called a biographical application blank (BIB). rests on the assumption that past behavior predicts future behavior and measuring this helps indirectly uncover motivational characteristics of applicants. can get lots of info in short amount of time from applicants. BIB has very good criterion related validity and minimizes adverse impact, even just a few biodata items can make a positive difference. can especially be useful to prescreen applicants when your selection system involves expensive procedures such as assessment centers how to develop: 1. select a job: can be applied to high or lower levels jobs, but keep in mind that ROI may be lower for lower level jobs 2. analyzing the job and defining the criterion life history domain: conduct job analysis and do literature search to determine what antecedent behaviors and life experiences may account for performance. FJA is especially useful for developing BIBs. 3. forming Hypotheses of life history experiences: in relation to the criterion. 4. develop a pool of biodata items: select or construct biodata items to reflect experiences that were hypothesized in previous step, can use already made items from biodata item sources in publications 5. pre screen and pilot test items: review by SMEs for bias and try out of representative group of respondents at least 300 preferably 500; remove items as necessary 6. scoring the biodata form: develop how they'll be scored, any method you use should be cross validated. can use single score or multiple scores for dimensions of items on the inventory.

Answer 30

1. deal with past behaviors/experiences 2. avoid items of personal nature 3. be specific but brief as possible in question and response options 4. escape option should be given 5. measure unique and discrete external events to avoid social desirability in responding

Answer 31

items exhibiting little response variance items w skewed response distributions items correlated w protected group characteristics items having no correlation with other items thought to be measuring the same life history construct items not having validity with criterion measure

Answer 32

use critical incidents JA technique to develop questions | incidents are sorted into themes of similar behaviors by SMEs

Answer 33

use critical incidents JA technique but also identify each dimension as essentially describing either maximum or typical performance. maximum = deals with technical skills and knowledge; typical performance dimensions should be used exclusively for behavioral questions since interviews can be primary way of measuring OCBs. use probes/follow up questions and recognize distinction that should be made between applicants who have work experience related to the job vs. those who don'. For people who don't have experience, questions should be framed more general and broadly to score each dimension is scored separately and given a weight reflecting its importance to overall job performance. equal weights should generally be used unless a dimension is at least 2x more important than the others to job performance. place applicants in one of 5 rank order groups for each dimension then sum their dimension scores.

Answer 34

restrict the use of the interview to the most job relevant KSAs relating to: job knowledge, applied social and interpersonal skills, and personality behaviors limit the use of preinterview applicant data: better for validity and objectivity. information that would be okay to look at beforehand is info relating to the KSAs to Be covered in the interview or inconsistencies on the application blank or resumes adopt a structured format: formulate set of questions for each KSA in the interview use job related questions use multiple questions for each KSA rely on multiple independent interviewers: increase reliability, use same across candidates apply formal scoring format: preferably behaviorally anchored rating scales based on critical incidents representative of the dimensions and sum them train the interviewer: receiving information, evaluating information, and regulating behavior

Answer 35

concerns of females, disabled, and older workers; adverse impact an issue for these groups the test must clearly be linked to critical job tasks that require physical abilities in their completion. if tasks can be modified to reduce or eliminate physical demands for tasks, they should be and the test reevaluated for use

Answer 36

review reliability data: make sure reliability studies done on the test are sound, have a big enough sample, etc. A good reliability coefficient is .85-.9. this means test scores are likely to be closer to an applicant's true score and the test results are more precise review validity data: correlation of test scores with other measures and factor analyses of test items

Answer 37

conscientiousness and emotional stability predict performance across most jobs one problem is that their predictive validity is still generally quite low, rarely exceeding .2. our theoretical understanding of why some traits are important for some jobs is still inadequate but evidence has been offered evidence that supports using personality to make selection decisions especially with the growing consensus of the Big Five dimensions. also, managers like using it and view it as being nearly as important as GMA. even if personality only moderately predicts performance, it can help with a range of other things, like identifying people likely to turnover, engage in OCBs/CWBs, teamwork and leadership, outcomes managers care about. personality traits contribute incremental validity as they don't highly correlate with other useful selection tools and have little to no adverse impact. the effects of personality are longitudinal, predicting outcomes years later. this demonstrates their usefulness.

Answer 38

Measurement methods: reflect multiple KSAOs simultaneously Criterion related validity: is strong Subgroup differences: are small to moderate

Answer 39

Construction: new ways besides CIT to develop situations and items Reliability: better methods than chronbach’s alpha

Answer 40

Coefficient alpha is most utilized and reported form, but is not the most appropriate for SJTs. There is currently a contradiction in the literature regarding the appropriate method for reliability for SJTs. Some believe that internal consistency is inappropriate given SJTs are usually multidimensional in nature. Yet every published study on SJTs reports internal consistency. The issue of reliability for SJTs is important given they are often used for high stakes decision making. Lower internal consistency reliability common to SJTs makes it hard to develop cut off scores/minimum standards when using them. Thus, test retest reliability is often considered the more appropriate approach because multidimensionality does not need to be considered.

Answer 41

Use CIT gather from SMEs using the antecedents-behavior-consequences method. Antecedents will be used to construct the situation, while the behaviors and consequences are used in the construction of alternative response options. Each item is created to represent actual instances of behaviors in the workplace.

Answer 42

the rational approach to designing the answers uses the judgment of SMEs and the test developer The continuous method of scoring that key allows for a range of possible scores per item. You assign a scheme ranging from -1 to 1 (worst to best answer) to all items.

Answer 43

To elicit more knowledge-based responses, item stems ask what the respondent should do, or which answer is the best. To elicit a more behavioral response, ask what the respondent would do or would most likely do or has done.

Answer 44

Most of the SJT research has followed Motowidlo et al 1990’s approach to development. This included the use of CIT to develop situations, the use of rational key and multiple choice response format, and continuous scoring. Continuously using this approach means that we may be missing out on something better. We need to figure out if there are better ways to develop SJTs. Further, using CIT can limit generalizability to other settings because it is inherent to that specific job/situation. Would the same SJT with identical items have validity that gen- eralizes to different settings? Such validity generalization is possible for context-free measures of KSAOs such as cognitive ability or personality, but would it occur for an SJT? Additionally, most SJT research is conducted in field contexts that apply concurrent designs. Directly related to this issue is the fact that a primary concern in SJT research is the reporting of reliability as internal consistency; critical incidents techniques that are so often applied are bound to produce high item-specific variance, thus lowering reliability. Using more theory based approaches to SJT development is a good way of reducing this issue and enhance item homogeneity. Lab studies need to be done more in SJT research, perhaps for testing theoretical frameworks to complement field setting research. Research could also look at organizational influences on SJT scores (such as org culture), since SJTs are contextualized by nature.

Answer 45

Use JA to identify technical and non-technical aspects of performance Use cognitive and non-cognitive predictors to measure full range of KSAOs supplement cognitive predictor with alternative predictor methods (ones that measure more than one construct) Minimize verbal/reading requirements to reduce cognitive load of predictors to the extent supported by JA Enhance applicant reactions via using sensitivity review panels of SMEs or simply giving explanations for why a certain measure or procedure is being used Consider banding

Answer 46

Interview structure typology consists of content (things like standardization of questions, basing questions on JA, etc.) and evaluation dimensions (using anchored rating scales, notes, same interviewers across applicants, etc)

Answer 47

Incremental validity over personality and GMA tests because they are only weakly related to each other Can predict ethical behaviors, or maximum or typical performance

Answer 48

Not all impression management is bad; if it is job related, such as with customer service jobs, it could be assessed during an interview and be useful information

Answer 49

Situational: intentions predict future behavior; asked to describe what they’d do in a hypothetical job related situation, usually a dilemma; get at job knowledge or even GMA Past behavior questions: past behavior predicts future behavior; asked to describe what they did in past job related situations; get at past experiences Have similar reliability and validity estimates, can complement each other if used correctly

Answer 50

agreement that structured is more valid than unstructured, but still unclear on how to define “structure” and when it is defined, unclear whether certain applications of structure adequately reflect operationalization. Needs to be more research on structured alternative interview techniques such as phone or video since these would likely benefit from even more structure than in person. Impression management also needs more research in terms of when it’s job relevant how it affects validity of the interview. Research should also look at structured interviews as an alternative way of assessing personalities over self report measures.

Answer 51

Acs have issues with construct validity. 25 years of research have indicated that the ratings that are completed at the end of each exercise are not reflections of the actual dimensions intended to be evaluated, but the exercises themselves. Evidence suggests that this discrepancy is due to the lack of cross-situational consistency in candidate behavior. One suggestion to alleviate this is to design Acs as task or role based, rather than traditional dimension based. Acs have demonstrated strong criterion and content validity, but the issue of construct validity is pervasive; they tend not to measure what they were designed to measure. Lance (2008) points out that this is largely due to a misapplication and misuse of the MTMM matrix in demonstrating construct validity. Post exercise dimension ratings are used to form final dimension ratings and summary ratings resembles an MTMM design, with dimensions representing traits and exercises representing methods. This method has formed the basis of Acs and AC research in the last few decades, and operates under the theory that AC dimensions are stable categories that are distinct within exercises and consistent across exercises, thus SDDE (same dimension different exercise) correlations should be stronger than DDDE correlations. But often this is not the case, as DDDE correlations are almost always larger than SDDE. We should focus less on behavioral dimensions and more on roles or tasks in exercises via simulations or work samples. More accurate assessment of behaviors, fewer inferences needed, red complexity of rating process and cognitive load on raters because behaviors are more proximal to the actual job.

Selection (General) Flashcards

(75 cards)