HireLabs approaches the reliability of a test from the context of the extent to which the variation in test scores is due to true differences between people on the characteristic being measured (e.g. abstract reasoning ability, extraversion, etc.) or to random measurement error.
The fact that a test is reliable only means that the test is consistently measuring a construct, it does not indicate what construct the test is consistently measuring. The concept of validity addresses this issue. As Kline (1993) notes "a test is said to be valid if it measures what it claims to measure".
At HireLabs, Reliability is generally assessed using one of two different methods; one assesses the stability of the test's scores over time, and the other assesses the internal consistency, or uniformity, of the test's items.
It would be important to note that a test's reliability sets an upper bound for its validity. That is to say, a test cannot be more valid than it is reliable because if it is not consistently measuring the construct it cannot be consistently measuring the construct it was developed to assess. Therefore, when evaluating the psychometric properties of a test, its reliability is usually assessed before addressing the question of its validity.
This method for assessing a test's reliability at HireLabs involves determining the extent to which a group of people obtain similar scores on the test when it is administered at two points in time.
Also known as uniformity of items, this method for assessing a test's reliability at HireLabs involves determining the extent to which, if people score well on one item, they also score well on the other test items.
In 1978, the Equal Employment Opportunities Commission (EEOC) created guidelines to ensure that the knowledge gained from testing is applied with impartiality to protect minority applicants from discriminatory employment procedures. Therefore, HireLabs adheres to these guidelines to ensure that the tests that you, the customer, administer are fair and balanced.
The Equal Employment Opportunities Commission (EEOC) guidelines do not state that one method is better than another; the method used must fit the needs of the business or organization.
There are three means of validation set forth by the EEOC:
- Construct
- Criterion (Concurrent and Predictive)
- Content
The term construct is a technical term for personality traits like intelligence and creativity. Construct validity is demonstrated if a test measures traits that have been found to influence successful performance of a job. That is, construct validity studies such as these demonstrate the "identification of construct(s) believed to underlie successful performance or those critical or important work behaviors in the job or jobs in question"[1] and their manifestations in the applicant.
An example of a construct, in this context, may be the ability to clearly communicate, such as communicate goals, project parameters or motivate. Such a test would be suitable to administer to an applicant for a project management position.
HireLabs ProfileSense™ assessments participates in construct related validity studies on a case by case basis.
If data demonstrates that a test is significantly correlated with a vital measure of job performance, the test is said to demonstrate criterion validity. If all the current managers that scored highly on a selected test to measure project management skills completed their projects on time and under-budget, the test would demonstrate criterion validity.
A criterion related test may be developed for a sales position, if, for example, it is determined through the HireLabs OccupationDNA that the position description could include monitoring, critical thinking, and persuasion.
Criterion-related studies of validity must document that the empirical data gained by employment selection procedures are "predictive of, or significantly correlated with important elements of job performance"[2]
Because each job description for each organization is unique, rather than developing tests for each
position that you may have, HireLabs ProfileSense™ crafts objective assessments based on its ODNA talent profiling methodology that will aim to target specific desired skills. Of course, with the HireLabs ODNA, YOU get to pick and choose those skills that you would like to test. Criterion related validity does not fit the type of information gained from HireLabs testing and, hence, is not applied to HireLabs assessments.
Clients may, however, request HireLabs for help with specific validation of critical-to-job-success information based on their established norms and benchmarks.
Content validity is demonstrated if the questions that make up an assessment are representative of content that is required to perform a particular activity or task. A test made up of algebra questions given to an applicant for a math teacher's position would demonstrate content validity.
It is important to note that content validity is situational. While the internal content of a test may be valid, test validity may be compromised if administered under adverse conditions.
HireLabs is committed to adherence to EEOC Guidelines. It is the responsibility of the test administrator to ensure external validity. Please click here for more information on customer responsibility in validation.
Where validity studies cannot or need no t be performed, in such circumstances, the test administrator should utilize selection procedures which are as job related as possible and which will minimize or eliminate adverse impact.
The Employment Test Validation Process is usually accomplished through the use of a Rating Committee. The Rating Committee's role is to determine if a pre-employment test being used as job selection criteria is a valid measure of the knowledge, skills, or abilities (KSAs) and that it is being used to measure without creating an adverse impact against any protected group in the candidate population.
Rating Committees are composed of Subject Matter Experts experienced and familiar with the job for which a pre-employment test is to be validated. The Test Validation Steps are observed and the meetings and findings are documented.
Test Validation Steps involve the following:
- Determining the KSAs - HireLabs determines the KSAs to be measured by a written or performance test that are needed on the employee's first day of employment, necessary and important to job performance, and linked to one or more important or frequently performed job duties.
- Developing a Test Plan for Measuring the Selected KSAs - this includes establishing the general components of the test and choosing the number and type of test items.
- Developing Test Content - HireLabs develops the test content and thoroughly reviews the test plan before beginning work.
- Validating the Test - A second group of HireLabs subject matter experts convenes to review the contents of the test especially the questioning styles, difficulty, comprehension, job-relatedness, and undue complexities.
- Scoring and Analyzing the Test - HireLabs administers the test to the test population. Human Resources specialists analyze the test results, providing item-level and test-level analysis, and determines if the test can be declared valid.
- If and when the test has been validated, the final test score for each applicant is reflected in a field on the hiring matrix and incorporated into the overall assessment of each applicant.
2. EEOC Code of Federal Regulations. Title 29: Volume 4, Section 1607.5, pg. 204, Revised July 1, 2001. [29CFR1607.5]
