TranscriptHi there! Glad to see that you're willing to help us out with the survey. To get you started, I will briefly explain what questions you will come across and I'll also describe some of the tools that we provide to help you fill everything out.
The purpose of the survey? Establishing the validity of selection tests across different types of jobs. Or in other words... ...we would like to know how the correlations of certain predictors of performance vary with certain job characteristics. We will refer to these correlations as the Validity Coefficients.
A Validity Coefficient is the magnitude of the correlation between test scores [e.g., GMA or personality] and a criterion variable [in this case job performance]. The predictors in this study [i.e., test scores] include General Mental Ability and each of the Big Five personality factors. Job characteristics we distinguish include the degree to which workers in the occupation deal with data, with people, with things the extent to which they are constrained in their actions and the extent to which their decisions or actions have an impact or have consequences.
This is a sample question from the survey. In this question we ask what change you expect in the correlation between General Mental Ability and performance for a job in which the worker would deal with data are relatively often. In other words, a job that's "high" on data. What do we mean by "high"? For this survey we consider a job to be high on a certain characteristic if its rating on that characteristic is one standard deviation above the mean.
To help you in answering the questions we provide several tools. One of the key ways in which we try to help you with your estimate is by providing the mean correlation between the predictor and performance. And perhaps most importantly... we provide the Credibility Intervals.
The Credibility Intervals have been derived from meta-analytic studies. They indicate the values that lie one standard deviation above and one standard deviation below the mean for the Validity Coefficients. Other than with Confidence Intervals Credibility Intervals have been corrected for sampling error and as such, they're considered to be the true range within which the validity coefficients lie.
Why are these Credibility Interval so important? Well, since we ask you what the Validity Coefficient would be when the job characteristic rating is one standard deviation above the mean and since we know that a one standard deviation change in one variable cannot lead to a greater than one standard deviation change in another variable the estimate that you provide cannot be greater than one standard deviation above or below its mean. Otherwise the correlation between the Validity Coefficient and the job characteristic would be greater than one. And of course... a correlation cannot be greater than one. Thus, the Credibility Intervals are important because they specify the upper and lower boundaries of your estimate.
Note that this assumes that the relationship between the two variables - the Validity Coefficient and the job characteristic - is linear. And that the specified distribution, the mean, and the standard deviation for the Validity Coefficient is correct. But we ask you to take this as a given.
Besides helping you by providing the mean the Credibility Intervals we provide some extra information once you start entering your estimates.
Here you can see a question being answered. As the input field was selected the definition and a description of the job characteristic in question were provided in the info box below. Once you start entering a value or estimate, the magnitude of that value is visualized in the bell curve next to the question. This bell curve updates as you change your estimate. It indicates when the estimate constitutes a small, a medium, or a large effect. When the estimate falls outside the colored area of the bell curve it falls outside the Credibility Interval and is no longer considered to be valid.
Lastly upon entering your estimate we display the implied correlation between the job characteristic and the Validity Coefficient. We refer to this correlation as the "Second-Order Correlation". This correlation shouldn't be greater than one.
With this we've come to the end of this introductory tutorial. If you want to review this information or if you would like to see further explanations on the mechanics of the questionnaire or the related concepts, you can press the "Help" button at the top right corner of the screen and review the video, look at a walk-through of a sample question or review some frequently asked questions.
Good luck in answering the questions! And thanks again for your help.
Frequently Asked Questions (FAQ)
Here we have listed the most frequently asked questions and their answers. In order to view the answer to a question, please click the triangle () next to the question.
Submit a New Question
Estimates for Sales Managers
For the job Sales Manager, estimate the correlation between the various predictors and overall job performance. The chart provides the distribution of the correlation accross different jobs.
Sales Manager. Plan, direct, or coordinate the actual distribution or movement of a product or service to the customer. Coordinate sales distribution by establishing sales territories, quotas, and goals and establish training programs for sales representatives. Analyze sales statistics gathered by staff to determine sales potential and inventory requirements and monitor the preferences of customers.