ALBERT

Notes: The method of selecting among job applicants using statistically based banding has been proposed over the last 10 years as a way to increase workforce diversity. The method continues to be reviewed by academics and considered by practitioners. Although the goal of increasing workforce diversity is important, statistical banding of scores remains controversial. We present a set of unique, statistically and theoretically based criticisms of a form of banding (top-score-referenced banding) that is widely used in hundreds of jobs in the public sector throughout the United States. We suggest that even within the premises of such banding, the wrong formula is used to estimate the standard error of measurement and standard error of the difference. One consequence is that too many individuals are labeled as essentially equal with respect to test scores. A related consequence is that test scores within a single band are statistically different and should therefore be treated as such for selection purposes. A more logically and statistically defensible procedure for responding to diversity concerns is to continue to attend to adverse impact issues at each step of the recruiting and test development process.

Notes: Applied psychologists have long been interested in the relationship between applicant personality and employment interview ratings. Analysis of data from two studies, one using a situational interview and one using a behavioral interview, suggests that the correlations of structured interview ratings with self-report measures of personality factors are generally rather low. Further, a small meta-analysis integrates these two studies and the limited previous literature to arrive at a similar conclusion – there is relatively little relationship between structured interviews and self-reported personality factors.

Notes: Although there has been empirical attention paid to the criterion-related validity of predictor composites, there has been much less attention paid to the standardized ethnic group differences associated with these composites. One important area of inquiry in predictor composite research is the influence of adding predictors to a test of general mental ability. The limited empirical literature on this practice is mixed, but the prevailing expectation is that there is likely to be higher validity and less adverse impact. Unfortunately, much of the previous work is limited by the presence of inaccurate validity and standardized ethnic group difference values. In this analysis we formed meta-analytic matrices to more accurately estimate the validity and standardized ethnic group differences of several composites that combine a measure of cognitive ability with measures of conscientiousness, a structured interview, or biodata. While results were somewhat complex, we found that adding alternative predictors does not result in a situation in which validity automatically goes up and adverse impact potential automatically goes down. In fact, the reductions in adverse impact (if any) from adding “non-cognitive” predictors were more modest than much of the literature suggests.

Notes: Almost all book chapters, review articles, and textbooks in the field of personnel selection suggest that work sample tests are associated with lower levels of ethnic group adverse impact than paper-and-pencil tests of cognitive ability. However, the empirical literature is heavily dependent on adverse impact estimates obtained from incumbent samples rather than applicant samples. As such, parameter estimates are subject to range restriction from prior selection and on-the-job experiences. Further, an emerging consensus in the selection literature indicates that any method of assessment can be associated with high or low levels of adverse impact – depending on the nature of the construct(s) being measured. To begin to examine these issues, we present two recent sets of applicant data from public sector jobs (for a management and entry-level job, each with technical and interpersonal skill requirements) and show that adverse impact of work sample exams might be more extensive than realized. We discuss the mismatch between what the field of employee selection “knows” and what is said in articles/summaries about work samples. Employers and other practitioners who depend on advice in academic overview articles may be overly optimistic and eventually disappointed by minimal reduction in adverse impact. Implications for workforce diversity and future research needs are also discussed.

Notes: The primary purpose of this investigation was to test two key characteristics hypothesized to influence the validity of situational (SI) and behavior description (BDI) structured interviews. A meta-analysis of 54 studies with a total sample size of 5536 suggested that job complexity influences the validity of SIs, with decreased validity for high-complexity jobs, but does not influence the validity of BDIs. And, results indicated a main effect for study design across both SIs and BDIs, with predictive studies having 0.10 lower validity on average than concurrent studies. Directions for future research are discussed.

Notes: Abstract Attaining the ambient standard for tropospheric ozone has been difficult in many metropolitan areas, despite efforts to reduce anthropogenic sources of the ozone precursors, including the nitrogen oxides (NOx). Until recently, NOx emissions from biogenic sources in soils were not considered in simulations of air quality and emissions reductions scenarios, yet they may be significant, especially in agricultural regions where nitrogen fertilizers are applied. Soil NOx is produced primarily by microbial processes; production and emissions from soils are controlled by a suite of environmental variables, including inorganic nitrogen availability, water-filled pore space, and soil temperature. Agricultural management practices such as fertilization and irrigation affect these environmental variables and thus have the potential to dramatically alter soil NOx emissions. Although current models incorporate some of these variables, accurate regional estimation of soil NOx emissions requires modeling approaches that explicitly incorporate the spatial and temporal patterns of management practices, especially fertilization, as well as other environmental controlling variables such as water-filled pore space and soil temperature.

Notes: This review evaluates analyses that are or may be performed to estimate uncertainties associated with air quality modeling used in regulatory planning to meet National Ambient Air Quality Standards for ozone. The sources of uncertainties in photochemical air quality simulation models (PAQSMs) are described. Regulatory requirements for evaluating PAQSM performance and uncertainty concerns not addressed through standard performance evaluations are discussed. Available techniques for evaluating uncertainties are presented. Experiences with analyses conducted most commonly are reviewed, as are those that might be used in a cohesive model uncertainty evaluation. The review concludes with a call for renewed emphasis on applying current techniques complemented by heretofore sparsely used diagnostic, corroborative, and alternative approaches and enhanced observational databases.

Notes: Abstract A sufficient historical record in grid-based O3 modeling now exists to permit assessment of its role and limitations in policy analysis. In this presentation, several topics are examined: past experience in model evaluation and use, the present status of modeling, the value of modeling in policy analysis, and key issues and future needs. First, the role of grid-based O3 modeling in policy analysis was assessed through a case study of nearly 10 years of model applications in the South Coast (Los Angeles) Air Basin. Changes in quality of performance and in degree of acceptance of grid-based models (in policy analysis) with time were analyzed and compared. Degree of acceptance appears to depend on a variety of factors, including level of understanding and familiarity, perception of need, and relative degree of acceptability, as compared with other available models. Improvements in quality of performance with time were limited (and, in any event, knowledge of such changes were not available), and thus this factor seems to have had little or no role in influencing model acceptance. Second, the current state of modeling, in terms of both science and art, is appraised, considering the state of knowledge, availability of data bases, adequacy of performance evaluation procedures, and quality of predictive performance. Significant deficiencies have existed in knowledge and treatment of key governing processes in models. Until recently, data bases suitable for use in evaluating model performance were unavailable. Quality of performance has not been sufficiently good to confer confidence in models' use in a regulatory environment. Evaluative testing did not provide sufficiently for ‘stressing’ models. For many, but not all of these issues, attention is now being, or soon will be, given to alleviating concerns. Third, the role and value of modeling in policy analysis is discussed. Attention is given to the role of the modeling expert in the policy forum, as compared with the model itself, and with the consideration of modeling as a longer term process rather than as a direct or short term means of generating ‘answers’. Finally, key concerns and future needs are delineated, and some unsolved problems are discussed.