Management 614/814
|
Interview Validity and Reliability Measuring Applicant KSAs More Is Not Better: This issue is concerned with the number and type of applicant characteristics that interviewers often attempt to measure. It was not until recently the EEOC made it expressly clear that the selection interview was viewed as a selection test and must be validated before use. Before this clarifica-tion, the interview was used to assess a wide variety of applicant characteristics and the result was often superficial data of limited value. Even when the interviewer concentrates on only a few characteristics, there are some that are not profitably addressed. All too often a conclusion about the applicants mental ability, arrived at during a 30-minute interview, may be less efficient and accurate than one based on the administration of a 10-minute test. Appropriate KSAs: There are two main types of characteristics that are best assessed in the interview: Personal Relations, such as sociability and verbal fluency, and Good Citizenship, such as dependability, conscientiousness, stability, and perseverance. Personal Relations characteristics are important for successful personal interaction in short-term or initial meetings. The interview, by its very nature, is an example of such a situation and, therefore, should be an accurate indicator of ability in these areas. The Good Citizenship characteristics are often evaluated after discussion of work habits, tasks completed, and work environments. Such discussions usually require clarification or elaboration of statements made by the respondent. The interview, because of its interactive nature, is suitable for such probing by the interviewer. A third characteristic, job knowledge, has also been evaluated in interviews. One appropriate situation is when the applicant has serious reading or writing deficiencies that would impede selection evaluation but not job performance. The interviewer would be able to determine if the question is understood and to clarify unclear or poorly worded responses. Another condition would be selection situations for jobs that require the verbalization of technical information and work procedures, for example advisory or consulting jobs in which most of the requests for service are oral. In such situations, the interview approximates a job simulation selection device. Selection Evaluation by an Organization Member Questions Used: Logically, the nature of the questions asked are critical to the reliability and validity of the interview. If the questions are not related to the job, we should not expect validity to be high. If the questions used are not consistent among interviewers, we should not expect them necessarily to agree on their evaluations of applicants because each interviewer has, in fact, collected different information about the applicants. Applicant Characteristics: The effect of the interviewer’s personal liking of the applicant has consistently been found to be related to interviewers’ evaluations. The interviewer’s perception of similarity to the applicant in attitudes, demographics, and other characteristics is also clearly related to his or her evaluation of the applicant. However, it has been shown that this similar-to-me effect is much less pronounced when the interview is structured and job requirements are clear. Nonverbal Behaviors: Most studies have concluded that nonverbal cues are, in fact, related to evaluations. The cues investigated the most often are eye contact, head movement, smiling, hand movement, and general body posture (rigidity versus movement). It appears that high levels of these cues are related to favorable interview decisions. The studies also lend support to the idea that an interviewee can manipulate the outcome of the interview through learned behaviors. This obviously brings into question the advisability of using such data in making evaluations of applicants. Weighting Information: It has been
found that relatively larger weight has been given to negative information over positive
information in the interviewer’s decision, even for experienced interviewers.
Related work has found that interviewers could usually specify why a rejected applicant
was not likely to be a good employee but could not clearly articulate reasons why
acceptable applicants would be satisfactory, indicating a clearer use of negative
data. This emphasis on negative information has to do with the costs of
various
outcomes of the interview. The greatest cost to the interviewer is making the
mistake of selecting an applicant who fails on the job. Professional reputation and
company expenditures suffer. On the other hand, rejecting an applicant who would, in
fact, be a successful performer if selected has no real, tangible costs. Therefore,
cost is lowered by rejecting all marginal or doubtful candidates. Establishing goals for the system is the first step in developing an effective screening device. The goals will help guide the development of the interview system so as to ensure that it will benefit the organization. Select Position KSAs As mentioned earlier, a single selection device cannot accurately measure every relevant position KSA. As a result, KSAs that can best, or most effectively, be measured using the interview should be chosen. These KSAs would reflect the important requirements for the job that, to a high degree of certainty, can be measured using the interview system. To develop the interview system with the intention of having it measure every necessary KSA would not be a realistic expectation and could possibly lead to an inaccurate conclusion that the system is a failure because it was unable to measure every KSA, when, in fact, no single selection device can realistically and accurately measure every requirement for a job. Furthermore, the KSAs that are chosen should each be weighted to reflect its importance for successful job performance. Develop Standard Interview Format As mentioned earlier, one of the biggest problems of the traditional interview was the inconsistencies in the type of questions that were asked by interviewers. An effective interview system should have the interviewers ask a standard list of questions to every applicant so as to ensure consistencies during the scoring of interviews. The questions should address the requirements of the job only and be of the "closed ended" variety, in order to allow for easy scoring of the answers. Know Benchmark Answers in Advance In order to know whether an interviewee's answer is a good answer to a question, the interviewer must know in advance what constitutes a good answer. Allowing an interviewer to decide what type of response constitutes a good answer at the time he/she asks the question to the interviewee increases the possibility for subjectivity and interrater unreliability (if there is more than one interviewer). Therefore, every interviewer should be provided with a list of all possible answers to the standard interview questions so that each one will know what constitute good and bad responses. Develop Formal Scoring Format To ensure that all interviewees are scored on the same scale, the interviewing system should have a scoring system that will guarantee two interviewees with identical answers to a question or identical overall performances on the interview will receive the same score. Inconsistencies in scoring job applicants that cannot be attributed to business necessity can be a source of legal problems for employers. Consequently, development of the interview system should also include the development of a consistent and formalized method for scoring applicant performances on the interview. Develop Job Related Questions Another source of legal problems for an employer is the use of non-job related questions that can intrude upon the privacy of the interviewees. Many instances have occurred in which an interviewer has asked a question infringing on the rights of the interviewee as a member of a protected group, creating legal repercussions for the company. Even today, many interviewers still ask questions which are not only non-job related but are blatantly illegal as well. In order to avoid problems, all interview questions must focus on the requirements for the job and address a particular KSA. When arriving at a standard list of interview questions, great care must be taken so as to ensure that every single question can be tied to a specific job requirement and such a connection can be proven and documented. Any questions that cannot clearly be shown to be related to the job must be eliminated from the list of possible interview questions. Furthermore, the list of questions that are chosen for use during an interview should be occasionally reviewed by independent job experts to ensure the continued job relatedness of each question. Use Multiple Interviewers Using several interviewers, otherwise known as an interview panel, to question job candidates can improve the reliability of the overall interview process. However, this will only hold true if the interview is highly structured in which each of the interviewers are using the same list of questions and are instructed on how to score all interviewees. Furthermore, interrater reliability must be consistently monitored to ensure that the panel of interviewers can demonstrate agreement on the evaluation criteria. Train Interviewers As mentioned earlier, a successful interview system is highly dependent on all interviewers being well trained on system protocol. All interviewers should be trained on how to avoid allowing non-job related characteristics of the interviewee, such as mannerism, appearance, and personality, from affecting the interviewer's evaluation of that candidate. Although any biases that interviewers may possess for certain types of individuals cannot be completely eliminated, they should be reduced to a minimum when trying to objectively evaluate a job candidate. Furthermore, each interviewer should have a thorough knowledge of the position requirements and have a general understanding of the characteristics of the job that make for a successful performer. Use Job Simulations One of the big disadvantages to using a typical employment interview is that the interviewees are not given the chance to demonstrate the job related skills he/she may possess. In light of this drawback, the use of short simulations, lasting about 10 minutes each, that place interviewees in a situation that is very similar to those faced by the actual job holders and evaluating them based on their ability to handle such situations can greatly improve the job relatedness as well as the validity of the interview system. The interviewer can ask less questions and, instead, observe and measure the interviewee's critical job related behaviors. This can give the interviewer a better idea of whether or not the applicant can truly perform the job. However, once again, the mini-simulation, like a traditional job simulation, must be based on a thorough job analysis. Since they can be developed for virtually any job, combining such simulations with a well structured, standardized interview can make the overall interview system far more effective. Continuously Monitor System In order to evaluate the success or failure of the interview system, it must be closely and consistently monitored to see whether it meets the established goals. Monitoring the system includes ensuring constant legal compliance and checking to see whether actual correlation and reliability coefficients and the cost of the system meet the established levels. Measurements:
Additional Measurements
Reliability of Panel Interview
|