The validation of a measure of competency in the use of psychological assessment in career counseling: A Piagetian framework.

摘要

Based on the results of a prior field study, it was determined that an instrument that borrows from Jean Piaget's theory of cognitive development could be constructed and reliably used to measure assessor competence in the use of a career counseling assessment instrument in career counselor training. This research further explores the feasibility of validating this instrument. If successful, a training template could be created to provide competency measurement for the remediation of counselors in training and the improvement of counselor training models. The theoretical model upon which the instrument is based is the Piagetian Matrix of Test User Competence (PMTUC). The competency assessment instrument based on this theoretical matrix was named A Measure of Assessor Competence (AMAC). The AMAC produces one global score based on six test items. The long-term intent of this line of research is to promote the utility of the PMTUC in the creation of a variety of measures of competency (AMACs) across many psychological assessments. The PMTUC theory and the resulting AMAC instruments could be applicable to all instruments. The specific intent of this study was to validate the use of the AMAC in the creation of a measure of competency in the use of a career counseling instrument. The instrument selected for this validation research was the Career Thoughts Inventory (CTI) because experts in the use of this instrument were readily available. Therefore, the measure of assessor competency for this specific research study is the AMAC-CTI. Future studies might attempt to build measures of competency in the use of the MMPI-2 (AMAC-MMPI-2), Rorschach (AMAC-Rorschach), or perhaps the WISC-IV (AMAC-WISC-IV).;Studies 2, 3, and 4 involved expert raters, graduate students, and professionals in the field of counseling and career development. The graduate students were enrolled in a Combined Counseling Psychology and School Psychology doctoral program or the Mental Health Counseling masters program at a large southeastern university who have been trained in the use of the CTI. The professionals work in the field of counseling psychology and have also been trained in the use of the CTI. Participants were approached via face-to-face request, e-mail request, or telephone by either the primary investigator of this dissertation or the aforementioned Director of Clinical Training about volunteering for a study of trainee competency using assessments. Once persons agreed to participate, they were contacted via e-mail by the primary investigator and were directed via e-mail to access a web link provided by www.surveymonkey.com . Once participants accessed the link, they were introduced to the survey and presented with an electronic consent form and, upon agreeing to participate, a background questionnaire. Participants provided responded to six open-ended format questions which were assumed to correspond to the 6 primary determinants of test user competence. At the conclusion of the survey collection process, responses to surveys were redacted of personal identification information and given to expert raters to perform ratings using the AMAC-CTI.;For Study 2, inter-rater reliability coefficients and measures of internal consistency were derived to confirm the reliability of the instrument. An exploratory factor analysis (EFA) determined that the AMAC-CTI is a uni-dimensional instrument. Study 3 was conducted to examine the difficulty of the instrument. The open-ended portion of the survey required respondents to answer six detailed questions that corresponded to the six items that make up the AMAC-CTI. Based on the results of this research project, the performance tasks were determined to be somewhat difficult.;Study 4 assessed convergent validity by asking the student participants' clinical supervisors to rate their respective students' competency in the use of the CTI. Supervisors used the same evaluation criteria as the AMAC-CTI to assess their students. The student participants' overall AMAC-CTI scores were then correlated with the overall ratings provided by their respective clinical supervisors. It was hypothesized that these scores would be correlated, but statistical analyses failed to show a significant relationship. For Study 5, analyses were performed to examine the relationship between AMAC-CTI scores and education and between AMAC-CTI scores and experience in the use of the CTI. AMAC-CTI ratings were positively correlated with experience in the use of the CTI, but were not correlated with education level and the number of assessment courses completed by participants. Implications for further test development and counselor training of assessment skills are discussed.;To validate the AMAC-CTI instrument, five studies were conducted. Study 1 involved expert ratings of the importance of the six items in the AMAC-CTI using an Expert Content Rating Form. The experts in the field of counseling and career development were identified by the Director of Clinical Training in a Combined Doctoral Program in Counseling Psychology and School Psychology at a large university in the southeastern United States. For this study, persons were considered experts if they had at least 10 years experience in the field of career counseling, held faculty positions, currently supervised graduate students in career counseling, and had served as a supervisor for the student administration of at least two hundred Career Thoughts Inventories. All five experts agreed that the items on the AMAC-CTI were important to critically important.