During the past year we have all had to adjust the new norm of remote working. One of those big adjustments is how to proctor in a remote testing situation. The online version of the COPSystem VIA can easily be adapted to fit this need. Our recommendation is to use a conferencing application so that you can view the testing process while communicating with your clients. The share screen feature will allow you to monitor your clients responses while they are answering items and the video option will allow you to view them.
The easiest way to complete this process is to use the Add Examinee option to create the sign in for your client. This will allow you to control the sign in credentials and control when they sign into the system. If you are using a shared computer be sure to sign out of your account. Arrange the meeting with the client and instruct them to go to https://eap.edits.net. They sign into the system with the username and password that you created for them.
The only other part of the remote testing process that is a little different is setting up your clients with the CAPS Manual Speed and Dexterity Test (MSD). There are several ways to approach this with remote testing. If your client has a printer, the PDF for MSD can be downloaded and printed locally and you can time them during the meeting session. Once the test is completed the test page can be scanned and emailed or an image of it can be texted to you for scoring. With either method you then score MSD and enter the score for the client.
You do this by signing into your account and selecting the Results tab. Then from the Actions column on the right select the plus sign. This will open a dialog box that prompts you to enter the MSD score and at that time you can also change the norm group associated with the client.
The alternative work around for the MSD is to have your client select the option: I did not take the test. Then they are prompted to estimate their score by entering either Below Average, Average, or Above Average.
Once the remote testing process is completed Results will be immediately available to the counselor. Another good resource for information on Remote testing is available by accessing the presentation given by Mary Barros Bailey at the 2020 IARP Virtual Conference.
Correlations of inventories such as the COPS-P with other similar inventories provides information concerning the construct validity of the assessments. This technique is especially useful when developing and implementing new and/or alternative instruments. In the case of the COPS-P, which is an alternate form of the COPS Interest Inventory (COPS, Knapp & Knapp-Lee, 2015), correlations with the COPS clusters are useful in understanding the nature of the COPS-P scales. Correlations between the COPS Interest Inventory and the COPS-P are presented in the table below based on 153 college students.
Correlations between the conceptually similar scales of the COPS-P and the COPS range from .37 to .81 with a median correlation of .65. The correlations were highest between conceptually similar scales. For example, the COPS-P Science, Medical Life and Physical scales were most highly correlated with the COPS Science, Professional and Skilled clusters. This trend was evident for all the COPS-P to COPS scale correlations. The only conceptually similar scales that did not correlate as highly as expected were the COPS Service, Skilled cluster with the COPS-P Service clusters. However, this is not surprising as the COPS-P is used by college students that have self-selected out of all the occupations within the Service, Skilled cluster.
As might be expected in every instance the COPS-P scale is more highly related to the corresponding COPS Professional level scale than the Skilled level scales. The correlations between all the conceptually similar scales provide evidence that supports the validity of the two instruments.
One of the main reason’s companies use aptitude testing is to make better hiring and promotion decisions. Tests are often much better than interviews in predicting whether a person has the potential to do a job well. When designed properly, aptitude tests can fairly and objectively compare the potential of different candidates. One of the most important aspects of a test is that it is both valid and reliable, so you can be assured of a fair process and that your hiring practices are legally defensible if challenged. The Career Ability Placement Survey (CAPS) is a comprehensive, multi-dimensional battery designed to measure abilities related to performing a job. Typically, this test is used in conjunction with values and interest assessments in the COPSystem VIA package, but it can also be used by organizations to support hiring decisions. The following presents a case study of an organization that used the CAPS for employee selection and shows that it differentiated between high and low performers.
Job performance is one of the most important outcomes at work and has been defined as the measurable proficiency of work behavior that is under employees’ control and contributes to organizational goals (Campbell, McCloy, Oppler, & Sager, 1993). Since performance ratings are associated with employees’ salary and promotion, studying their predictors is of high interest for both organizational researchers and practitioners. Schmidt and Hunter (1998) established mental ability as one of the best predictors of overall job performance. In another meta-analysis on the predictive validity of specific cognitive abilities for job performance, Bertua, Anderson, and Salgado (2005) distinguished between seven occupational groups (clerical, engineer, professional, driver, operator, manager, and sales) and their results showed significant validities for predicting job performance within all those groups. As mental ability has been shown to be a predictor of job performance, we expected that it would be a strong predictor of job performance ratings.
Research on the main aspects of mental ability has often consisted of the following characteristics: learning, problem solving, information processing, and reasoning. For example, Gottfredson (1997) described mental ability as “the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience” (p. 13). According to Jensen (1989), learning has occurred when a change in a specific response to a given stimulus, situation, or problem was observed. Problem solving can be defined as successfully transferring a given actual state to a target state by overcoming barriers (Dunbar, 1998). Information processing involves transforming information and creating new information (Oberauer, Süß, Wilhelm, & Wittman, 2003). Finally, reasoning “is a process which may occur at any point in a thought-movement and consists in the appreciation of likeness and differences between old experiences and a new situation” (Skaggs, 1930, p. 439). Each of these mental abilities can be measured individually to consider a person’s aptitude, but the optimal use is for them to be used as part of a comprehensive battery to measure ability level related to work potential.
The Career Ability Placement Survey (CAPS) helps examinees relate their current levels of ability to career clusters. It is a comprehensive, multidimensional battery designed to measure abilities that are related to performing a job. There are eight ability dimensions keyed to entry level requirements for occupations in each of the 14 COPSystem VIA Career Clusters. Measures included in the CAPS are mechanical reasoning, spatial relations, verbal reasoning, numerical ability, language usage, word knowledge, perceptual speed and accuracy, and manual speed and dexterity. Scores are combined from each individual test depending on skill requirements for each career cluster to determine level of ability within each career cluster. The Technology, Professional career cluster is comprised of scores from the mechanical reasoning, spatial relations, verbal reasoning, numerical ability, and word knowledge tests. The CAPS is a valid and reliable battery with test-retest reliability coefficients ranging from .70 to .95. To establish concurrent validity, correlations with the General Aptitude Test Battery were obtained and ranged from .63 to .80 between conceptually similar tests. Correlations were obtained between CAPS tests and grades in specific subject areas. These correlations ranged from .30 to .60 between the CAPS tests and the subject to which it was most closely related. For example, the CAPS numerical ability test had the highest correlations with grades in math. These results are significant and demonstrate the validity of the CAPS. Predictive validity studies show that ability scores are significantly related to subsequent career choice and may also predict job-performance associated with that career choice (Knapp, L., Knapp, R. & Knapp-Lee, 2009).
Employee data from an organization were analyzed to assess the effectiveness of the CAPS as a selection tool in predicting job performance. A sample of 128 employees were included in the analysis. All employees had completed the CAPS and had supervisor job performance ratings which were based on a scale of 1-5 (1 = highest performers, 5 = lowest performers; scores were then reverse coded so that higher rated employees received higher scores for analysis). Employees were categorized into one of three groups. In order to distinguish between high and low performers the top two tiers, rated as 1 and 2, were combined (N = 40), and the lowest two tiers, rated as 4 and 5, were combined (N = 29). Average rated employees, rated as 3, (N = 61) were their own category (see Table 1 for means and standard deviations of test scores). Predictive validity of the scores on the CAPS was assessed by comparing categories of employee job ratings based on their CAPS scores.
The CAPS scores are presented in two profiles. One profile the CAPS Ability Profile reports scores in terms of individual scores on each of the eight subtests. The other report, the CAPS Career Profile, compares the CAPS scores to probability of success in terms of ability within a career cluster. When examining the scores on each of the eight CAPS subtests, the highest rated employees had higher average scores on all the eight CAPS tests than the average rated employees and the lowest performers. Additionally, the employee data from the CAPS Career Profile were analyzed to determine how well the CAPS predicted success within a career cluster. All the employees listed job titles were classified within the Technology cluster. Specifically, examining the Technology, Professional cluster, the high achievers were all predicted to succeed in that occupational cluster whereas, the middle and low performers were below the cutoff score for success in that cluster. In addition, when examining the scores for each group across each career cluster the higher rated employees had uniformly higher scores than both the medium and low performers.
A linear regression was calculated to predict employee rating scores based in Technology Professional scores. A significant relationship between employee rating and Technology Professional scores was found, F(1, 127) = 88.302, p < .001 (see Table 2). Technology Professional scores accounted for 41.2% of the employees’ performance rating (see Table 2). Technology Professional scores significantly predicted employee rating, as Technology Professional scores increased so did employee rating scores. A univariate analysis of variance compared employees rated in performance levels 3-5 as compared to employees rated at levels 1-2 on the Technology Professional cluster. The analysis showed a significant difference between the groups ranked as lower performing as compared to the groups ranked as higher performing F(1, 128) = 67.14, p < .001 (see Table 3). The employees that had performance ratings of 1 and 2 were significantly higher than those rated at 3,4,5 showing that the CAPS is a good predictor of success on the job. A second analysis showed a significant difference between high rated employees (1-2) and average employees (3). High rated employees scored higher on the Technology Professional cluster than average rated employees, F(1, 128) = 26.458, p < .001 (see Table 3).
The purpose of the current study was to evaluate the CAPS when used as a selection tool for employee performance. An organization provided employee performance scores as rated by the employee’s supervisor, and CAPS scores for each employee. The current study provides evidence that the CAPS can be used to predict job performance, specifically when the individual tests are combined to match a profile that aligns with the ability levels and requirements of job categories. Analysis of the employee performance data and CAPS scores show additional validity that the CAPS measures job-related abilities that are relevant and applicable to the Technology, Professional career cluster, and can be used in other settings such as different career clusters.
Bertua, C., Anderson, N., & Salgado, J. F. (2005). The predictive validity of cognitive ability tests: A UK meta‐analysis. Journal of Occupational and Organizational Psychology, 78(3), 387-409.
Campbell, J. P., McCloy, R. A., Oppler, S. H., & Sager, C. E. (1993). A theory of performance. Personnel selection in organizations, 3570, 35-70.
Dunbar, R. I. (1998). The social brain hypothesis. Evolutionary Anthropology: Issues, News, and Reviews: Issues, News, and Reviews, 6(5), 178-190.
Gottfredson, L. S. (1997). Why g matters: The complexity of everyday life. Intelligence, 24(1), 79-132.
Jensen, A. R. (1989). The relationship between learning and intelligence. Learning and Individual Differences, 1(1), 37-62.
Oberauer, K., Süß, H. M., Wilhelm, O., & Wittman, W. W. (2003). The multiple faces of working memory: Storage, processing, supervision, and coordination. Intelligence, 31(2), 167-193.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological bulletin, 124(2), 262.
Skaggs, E. B. (1930). The essential nature and definition of reasoning. The Journal of General Psychology, 3(3), 435-442.
Table 1. Means and standard deviations for the two CAPS cluster scores of Technology and of each CAPS subtest score by employee rating.
Table 2. Regression results of employee performance predicting Technology Professional scores.
Table 3. ANOVA results comparing high vs low rated employees, and high vs average rated employees.
We are excited to announce that the new version of the EdITS Assessment Platform (EAP) launched on March 14, 2019. We enhanced the user interface and added new features that make it easier to use.
Replaced the Manage Button
We replaced the “Manage” button with icons. You can now click the icons to access your group content. You can also click directly on the group name, or products to open the information or available products screens.
Add Examinees from within your administrator dashboard
Now you can add examinees from your group examinee list. From your dashboard, click the Examinee Icon to open the group examinee list. Click on the tab that says, “Add Examinee.” Complete the registration page and they are ready to test. Remember, the examinee must sign in using their username and password to take their test.
Changed Tokens to Passcodes
With our new passcodes, you get the same features of the tokens, with the added benefit of having multiple products within each group. To view your new passcodes, click on the “Products Icon” from your dashboard, or click on the link in the products column. This will open a list of your available products in that group. Each available product has a passcode associated with it. You can use this passcode the same way you used the token, when an examinee signs up, they will be able to enter a passcode and take one of your tests.
Please remember, that your old tokens have been deleted and they will not work. You must use your new passcodes moving forward.
One of the benefits of using the complete COPSystem VIA is that you have full access to all of your scores to use in comprehensive career exploration. The results page of the COPSystem allows students to review their top three career matches to pursue based on their highest rated values, interests, and abilities. Additionally, students are shown the abilities where their strengths would be useful in a career if they wanted to explore that path, even if it was not one of their highest interest scores based on the completed survey. This can be especially beneficial for students exploring career paths as it gives them more options to research. It can be the case that students don’t realize their interest in an area until they discover that they are good at it. The CAPS results help students discover how high school classes can help them reach their career goals. While it is certainly helpful to identify the top two or three areas of interests and strengths for a student, it is not ideal to limit them to those career paths when there are more areas that can be pursued.
Furthermore, measuring values provides students with a more complete picture of the type of career that they think will be the best fit for them. The COPSystem is the optimal tool for career exploration as it provides students with a comprehensive and clear picture of career exploration possibilities all within the same easy to use and interpret online platform.
In a recent study, data were collected from 611 high school students in the Midwest who were participants in the college and careers program. Scores from another company’s interest assessment (similar to the Interest assessment offered in the COPsystem VIA) and the CAPS ability battery were compared to evaluate the effectiveness of matching abilities and interests using the two tests as a tool for career guidance. Only the top two scores from the other inventory were available, whereas the CAPS reports ability scores for all 14 career clusters. The CAPS provides probability of success cut-off scores for each career cluster. Any score above the cut-off score results in a plus, or ability match for a student, which indicates that the student would be able to complete job related tasks adequately based on their current ability level. The advantage of the CAPS is that students look at both their strengths and weaknesses in terms of ability and decide whether they would like more training to improve some of their lower scores or if they would like to pursue a career aligned with their strengths. It is of great help in planning for high school courses and future career pathways.
Interest clusters from the other inventory were compared to the COPS interest clusters so that scores could be matched on like scales (See Appendix for a comparison of U.S.O.E. and COPS interest clusters). Once similar clusters were identified from each assessment, CAPS ability scores were used to evaluate whether the students’ abilities and interests were considered a match. A “Professional” match is for students who would be encouraged to pursue a college degree, and “Skilled” match is for students who would be encouraged to enter the workforce or pursue trade schools.
Data from the 611 students were analyzed to determine the effectiveness of the CAPS battery in matching interests and abilities. In the current sample, 100% of students had a match between their first ranked career interest, and their CAPS ability score. Of the 611 students, 135 had matches falling within the Service Skilled, Outdoor, or Technology skilled clusters which all contain job tasks that can be performed at relatively low ability levels, so the cutoff for these clusters is generous. For further analysis the 135 students from these matches were not included. Of the remaining sample of 476 students, 262 (55%) had a match between their first highest rated interest and abilities associated with a “Professional” career cluster; meaning they would be encouraged to pursue a college degree or post high school education.
The remaining 214 students (45%) had a match on their first highest rated interest with a “Skilled” level cluster; meaning they would be encouraged to pursue careers in career and technical education (See Figure 1).
Figure 1. Pie chart depicting percentage of students who would be encouraged to pursue college vs. enter the workforce based on CAPS Career Cluster match.
For the second highest interest match, 258 students were excluded because either the interest scores were missing, or they were part of the group previously discussed. Of the remaining sample of 353 students, 251 (71%) had a “Professional” match on their second highest interest score, and 102 (29%) had a “Skilled” match (See Figure 2).
Figure 2. Pie chart depicting percentage of students who would be encouraged to pursue college vs. enter the workforce based on CAPS Career Cluster match.
It should also be noted that there is some overlap between the 1st and 2nd highest interest match and students encouraged to pursue college, such that the highest performing students in the ability battery scored on average higher across all career clusters in ability levels. There were 184 students that had “Professional” matches on their 1st and 2nd career clusters, but they were not singled out in this analysis because we were interested in the assessing the CAPS at an aggregate level instead of on an individual level.
The CAPS ability battery when used in conjunction with an interest inventory from another company does provide students with career guidance that can be used to determine a path forward. Research shows that career satisfaction is related not only interests and abilities, but values as well (Knapp-Lee, 1996), which is included in the COPSystem VIA. This creates a more comprehensive and clearer picture of where students can focus their career exploration.
Combined with the Career Briefs, the COPSystem VIA is a comprehensive career exploration package that allows students and counselors to explore more career options and find the best fit for them. This easy to use platform gives students access to their full data including values, interests, and abilities and allows them a complete and more wide-ranging career exploration which leads to greater success in finding the ideal person-job fit.
Knapp-Lee, L. (1996). Use of the COPES, A Measure of Work Values, in Career Assessment. Journal of Career Assessment, 4, 429-443.
We are excited to announce an updated version of the EdITS Assessment Platform (EAP) launching on March 14, 2019.
EAP users can expect:
A more user-friendly interface
The ability to add examinees directly from the Administrator Interface
Screen reader accessibility for people who are blind
Replacement of the group token method with product passcodes
As part of this deployment, the EAP will be offline between 3pm – 9pm PST Wednesday, March 13th, 2019, so please adjust your testing accordingly. Usernames, passwords, results, and all other data will remain unaffected. However, group tokens will be replaced with a new product passcode.
Don’t forget to check out our new website and subscribe to our newsletter to stay in the know and receive the latest updates.
Please contact our support team with any questions or concerns.
The COPSystem is Unique It is one of the few assessment packages that includes the measurement of three components that are critical to career exploration, selection, and satisfaction. The COPSystem measures:
Values – what is important to you?
Interests – what do you like to do?
Abilities – what are you good at?
Valid and Reliable The COPSystem has been in wide use for over 50 years with a proven history of research that supports its accuracy and usefulness in career exploration and choice. Over 65% of the time the COPSystem results match a future college major or occupational choice. A summary of the reliability and validity is available here.
Easy to Use The COPSystem is easy to administer and easy to explain to clients so they can understand the results. The results are organized into logical career clusters that link to an occupational database of over 1,300 job descriptions that include training, salary, expected outlook, and links to both O*Net and live job postings powered by Indeed.com.
Has a Wide Variety of Options to Meet your Client Needs The COPSystem is a completely accessible package that is available as a web-based application or also available in paper/pencil in those situations where there is limited internet access. It is also available in Spanish translations for both paper and online formats. It is completely accessible to screen reader applications for people that are sight impaired. The inventories are written at a sixth grade reading level.
Works for all Educational Levels Norms are available for educational levels of grade 7-12, college and adult. Other forms are available for school children and non reading adults.
Results That Work The COPSystem is the optimal tool for use in the career exploration. When clients review their results they have the “oh wow this is me” experience that makes the entire process useful, worthwhile and enjoyable. It is educational and allows test takers to learn something about themselves and their options for many career paths in the process.
The internet is an essential feature of modern life used to check the news, sports, weather, engage in social media, and find employment, but for people that are blind, navigating the internet can be very difficult. It is essential that the internet be accessible to people that are blind so they have the same access to materials and resources that are crucial to daily living, and for them be more actively engaged in society. EdITS has adapted the COPSystem career guidance program to be accessible to individuals using a screen reader.
The National Federation of the Blind estimates that as many as 10 million Americans are blind or visually impaired, and in 2015 they estimated that only 42% of working age adults with significant vision loss were employed. Adapting the COPSystem to be accessible for using a screen reader is important to enable this population to seek a career path that fits their career profile. At EdITS, we are assisting in providing more employment insights, thus lowering the unemployment rate among people that are blind.
The challenge in making the COPSystem accessible for screen readers was in adapting the CAPS ability battery. There are eight timed tests in the CAPS, and all but two of them were able to be directly adapted for a screen reader. In an effort to accommodate low vision and sight impaired people a study was conducted to determine if a person who is blind could answer Spatial Relations test questions. It was found that this was not a task that could be done by the sample that were blind, so an alternative scoring option is incorporated in the screen reader adaptation. This is the most reasonable option for a person who is blind and will lead them to considering jobs that match their interests and abilities. In addition, the Manual Speed and Dexterity test has a series of self-report questions and a way to estimate the MSD score that enables a person that is sight impaired to complete the COPSystem.
As a part of this process we have made the CAPS available with increased time increments to make accommodations for special needs populations. The CAPS can now be purchased with the option of either time and a half (7.5 minutes per test) or double time (10 minutes per test). If you are interested in using either of these options our administrative team can assist you with choosing the option that best suits the needs of your clients. The results pages are all completely accessible as well, that enables people with a wide range of disabilities to access our career guidance program.
It is our hope that having these tools available will increase the quality of life for people who are blind, in the satisfaction that can be gained with a fulfilling job that fits each individual’s career needs. We are extremely proud to be helping the blind community by making our platform fully accessible to them.
Norms for the COPSystem VIA are based on data gathered from 2015 through 2018 for a sample of 32,638 sixth through twelfth grade educational levels. The National sample was aggregated based on a weighted population of subjects from four geographic regions of the U.S. identified by the 2010 U.S. Census Report. The average age of the sample was 29 years old. Table 1 presents a summary of the sample sizes from each region by gender. These groupings were used to explore possible mean differences in vocational interests, values, and abilities as a function of where they live and attend school.
Table 1. — Summary of Normative Data by Region and Gender
Consistent with previous normative analyses, comparisons were made across gender, grade level, and region to explore which of these factors, if any revealed mean differences of importance. Of these factors, only gender revealed large and consistent mean differences. This suggests that scale means for males and females should be interpreted differently, as has been done in the past. Thus, all four regions were aggregated at the national level and two normative distributions, one for each gender, were determined for educational levels 6 through 12.
Similar analyses were conducted on the COPES Values and CAPS Abilities inventories. As in previous years, separate normative distributions were produced only when mean differences were observed that strongly and significantly influenced the interpretation of scores.
Normative analyses of the CAPS Abilities battery revealed mean differences as a function of grade level. The student profiles reflect this trend, such that grades 8-9 are combined separately from grades 9-12.
The multiple profiles discussed here have been developed using best practices methodology. Our goal throughout this process has been to increase the overall meaningfulness and effectiveness of the recommendations made by those who rely on the COPSystem VIA to assist them as they guide and influence examinees during their career exploration process.
A career guidance assessment program is a valuable way to increase self-awareness and professional competence among students and working adults. The COPSystem consists of the COPS Interest Inventory, the CAPS ability battery, and the COPES work values survey. All three are long established, reliable, and valid assessments relating interests, abilities and values to occupations and occupational information. Reliability and validity information is presented in this article.
In terms of reliability for the COPS, alpha coefficients for each scale range from .83 to .91. Numerous studies have been conducted to establish the construct validity of the COPS. Correlations of the COPS interest scales to other similar assessments have been used to establish concurrent validity. In comparing the COPS with the Kuder, correlations between conceptually similar scales ranged from .21 to .49 which were significant and were as high as would be expected. In this study, 89% had at least one of their top three areas of interest the same on both the COPS and the Kuder.
When compared to the Holland based Vocational Preference Inventory (VPI), correlations between conceptually similar scales ranged from .50 to .70. To add further support to the construct validity of the COPS, declared major of entering college freshmen was compared to COPS scores and for 71% of the sample, declared major matched one of the top three measured interests. A long-term predictive validity study showed that 64% of students were in a job or college major that matched one of their three highest interest areas from one to seven years after taking the COPS.
The CAPS test-retest reliability coefficients ranged from .70 to .95. To establish concurrent validity, correlations with the Differential Aptitude Tests (DAT) were obtained and ranged from .65 to .81 between conceptually similar tests. Correlations were obtained between CAPS tests and grades in specific subject areas. These correlations ranged from .30 to .60 between the CAPS tests and the subject to which it was most closely related. For example, the CAPS numerical ability test had the highest correlations with grades in math. These results are significant and demonstrate the validity of the CAPS. Predictive validity studies show that ability scores are significantly related to subsequent career choice.
With regards to the COPES, alpha reliabilities ranged from .60 to .85. Concurrent validity studies demonstrated that the scales on the COPES are correlated .40 to .60 to conceptually similar scales on the DF Opinion Survey and the Allport Vernon which are as high as would be predicted. The COPS scores were compared to the COPES scores to confirm the validity of the values scores as related to the interest clusters. This analysis explored the relationship between interests and work values by selecting a sample of examinees with scores in a single interest area at the 75th percentile or higher and establishing a COPES profile.
The findings demonstrated the relationship between work values and interest scores that is reflected on the COPSystem Comprehensive Career Guide. A preliminary follow-up study for the COPES found an 89% hit rate for work values matched to subsequent career or college major choice. This article is a brief summary of technical information. For a more comprehensive discussion of the reliability and validity of the COPSystem please see the individual Technical and/or Examiner’s Manual for each assessment.