Friday, May 13, 2011

Validation and the 360 Solutions Assessments -Personality profile

There are many schools of thought regarding testing and validation. Some believe that construct validity is the most preferred method, others say criterion validity is the best approach. Still others say that a combination of both is the best of all possible worlds. We believe that common sense, good business sense and concurrent validity on a local level is the best approach. Why?


First, the goal is to find the best possible person for the position. Most hiring managers will say: "If I could ‘clone’ my best employee or employees in the same or similar positions, I would achieve my goal." Therefore, we attempt to identify the best candidates by a validation method known as "Concurrent Validity". What is concurrent validity? Quite simply, this is Benchmarking.
Benchmarking is achieved by selecting people with a common job description and evaluating them using the 360 Solutions Personality Profile. By grouping these individuals into Top, Middle and Bottom thirds, based on your standard measure of performance, it is possible to build a hiring benchmark using the Profile scores. The benchmark is based on a comparison of the top one third’s results versus the bottom. Additionally, we believe you should take this approach one step further and create your benchmarks on a local level.


Why do we recommend this approach?

Because what works in New York City does not necessarily work in Atlanta. What works in Dallas does not necessarily work in San Francisco. Many testing companies develop "norms" based on national averages of the top performers in a position. You then compare the test results of an applicant against those norms to determine if they will succeed in the position. To use an analogy, if my best performers in New York like ice cream, in Chicago they like deep-dish pizza, and in San Francisco they like sushi and I average them all together, what I get is a gastronomical mess. To further prove the point, practical experience shows us a top salesman in New York must modify his selling style in order to succeed in Atlanta. Why? Different markets. The Southern prospect may well reject the more aggressive selling style of the New Yorker. The same holds true for companies selling the same products or services. What works at Ford doesn’t necessarily work at Chrysler.

Second, the selection test must meet all EEOC and American Disabilities Act requirements. The 360 Solutions Personality Profile does.

Third, equivocation. Are you getting a true picture of the employee or applicant? The 360 Solutions Personality Profile has a consistency index. If the score is between 17-20 the results are considered 90% accurate to the individual tested. A score 0f 14-16 is considered 79-89% accurate. If the consistency score is below 14 and the tested individual is not an E type personality, the report should not be considered valid. If the tested individual has low confidence as measured by the Personality Profile confidence trait, this will affect the overall scores. If the confidence score is below 5, the report should not be considered valid.
Summary


In order to find the best candidate for the position, you must assess the candidate’s personality and skills to see if they "fit" local market conditions, office culture, management practices, and selling styles. How is this done? Through benchmarking. Your top producers are "best" in their positions because they have the skills to do the job and their personality is compatible with what you are asking them to do. By using the 360 Solutions Personality Profile and creating a benchmark around these top producers, you take into account the variables unique to your company and the affect these variables have on the human personality.

Statistical Analyses Supporting Benchmarking

In early 1997, a study was undertaken to see if there was any direct correlation between the 18 personality and character traits measured by the 360 Solutions Personality Profile and commissions earned on a national level. Profile scores from 64 companies across the country were gathered. Particular attention was paid to ensure that no one region of the country was more heavily weighted. Managers were then asked to rank their salespeople in terms of top and bottom one-third producers. Only salespeople with one year or more experience were included in the sampling. The criterion for ranking was commissions earned. The following character and talent traits were analyzed:
Stress Management     Outgoing Traits               Persuasive Level


Sensitivity                     Interaction                     Aggressive Level

Compassion                  Decision Making           Stubborn Level

Patience                        Analytical Setting           Goals

Creative Abilities           Independent                  Time Management

Artistic Abilities             Detailed Traits                Confidence


A single regression analysis was conducted for each trait against commissions earned. The highest r2 value achieved was .12. Second, a series of multiple regression analyses were conducted with as many as seven character and talent traits against commissions earned. Again the highest r2 value achieved was .16. Finally, a Pearson correlation coefficient analysis was conducted with each trait against commissions earned. The highest r-value achieved was .34. All of the above values show that there is at best a weak correlation between profile scores and commissions earned. Scores obtained on a national level would be unsatisfactory predictors of commissions earned


Conclusion

Other variables, such as local market conditions, selling styles, and management practices, as well as personality and character traits affect commissions earned. To further substantiate this, the following are the sales hiring benchmarks for two different companies. Both company benchmarks were established based on the top one-third producers’ results. The first company is located in the Midwest and the second is in the Southwest. Both have twenty or more salespeople and have been in business 10+ years. Both sell similar products.

©2010 - 360Solutions
It is apparent from these results; the benchmarks based on the top one-third producers in each company are very different. Yet, the products sold and incomes earned are very similar. Thus, benchmarking on the local level allows managers to have a method of "cloning" their most productive employees based on their company’s culture, management practice, selling style and other issues unique to their environment.


Determination of Accuracy

The same 64 managers were asked to rate the accuracy of the 360 Solutions Personality Profile on the top one-third and bottom one-third agents. As stated earlier, top and bottom were defined by the criterion of commissions earned as related by the managers. Additionally, only salespeople with one or more years of experience were rated. The average company employed 22 salespeople. A total of 896 salespeople were measured. The screening question asked was:

"On a scale of 0-100% how accurate was the 360 Solutions Personality Profile report on ______ (employee)?"

This question was asked during normal business hours. The manager made this rating within 24 hours of the reviewing the report, and before a joint review of the findings.

Accuracy of 360 Solutions Personality Profile

As Rated by Managers
Test Accurate % % of Managers Agreeing


95+ 23.4

1. 53.1

1. 10.9

1. 4.7

1. 3.2

< 75% 4.7

During this same period, tested salespeople were asked to percentile rank the accuracy of the 360 Solutions Personality Profile report findings. The screening question was:

"On a scale of 1-100%, how accurate was your 360 Solutions Personality Profile report?"

This question was asked during business hours. The salesperson made this rating within 24 hours of reviewing his or her report, and before a joint review of the findings.Test Accurate % % of Agents Agreeing

95+ 26.9

1. 51.0

1. 9.0

80-84 8.0

1. 3.0

< 75% 2.1

Summary

76.5% of company managers felt the 360 Solutions Personality Profile report findings had an accuracy rating of 90% or greater to their salespeople.

77.9% of the salespeople tested felt the results had an accuracy rating to themselves of 90% or greater.

Conclusion

Both the salespeople and the managers confirm the accuracy of the test. Therefore, it is logical to assume that local market conditions, selling styles, management practices and other environmental factors, as well as personality character and talent traits, affect the amount of commissions produced. Thus, the best possible method of selecting a salesperson that will succeed on a local level is through the benchmarking of the Personality Profile scores. This method takes into account the personality, character and talent traits necessary to succeed under variables specific to the candidate’s environment.

Final Note

The 360 Solutions™ Personality Profile with its unique benchmarking capabilities is the most cost-effective selection and development tool on the market today.

In addition to the profile results, this system provides additional important features to help improve the selection and development process:

1) In-depth interview questions are provided based on the Personality Profile score.

2) Additional skills and knowledge tests, including a Sales Aptitude Test and a Custom Aptitude Test

3) General intelligence and reasoning ability tests.

The results from the 360 Solutions Personality Profile System should not be viewed as the sole decision making tool in the employee selection process. Rather, they should be viewed as guidelines within an over-all selection system. The selection and development system should meet all legal requirements and provide other screening criterion such as background and reference checks.

The 360 Solutions™ Employment Testing System, properly utilized within the above guidelines, will help you to improve employee selection, reduce hiring mistakes and cut your turnover costs.
©2010 - 360Solutions

No comments:

Post a Comment