A Model for
Evaluating the Effectiveness of
Cognitive Reframes of Dysfunctional
Career Thoughts
Technical Report 36 - Revised
Darrin L. Carr, M.S.
June 26, 2004
Center
for the Study of Technology in Counseling and
100 S. Woodward Ave
The Florida State University
(850) 644-6431 (voice) (850) 644-3273 (FAX)
http://www.career.fsu.edu/techcenter
Copyright
© 2004 by
All Rights Reserved
Darrin
L. Carr is a Research Assistant, Instructor, and Career Advisor at The Career
Center and a doctoral student in the Combined Program in Counseling Psychology
and School Psychology at The Florida State University. The author wishes to
thank Drs. Gary W. Peterson, Robert C. Reardon, Janet G. Lenz, and James P.
Sampson, Jr. for their participation in this project. Inquiries and comments
should be addressed to Darrin Carr, at dcarr@admin.fsu.edu.
Table of Contents
|
|
|||
|
|
|||
|
Career Problem Solving, Indecisiveness, and Dysfunctional Career Thoughts |
|
||
|
Cognitive Information Processing (CIP) Theory: A Descriptive and Supportive Tool |
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|||
|
|
|||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|||
|
|
|||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
|||
|
|
|||
|
|
|||
|
|
|||
|
Appendix A: Systematic Cognitive Errors/Distortions Identified by Aaron Beck |
|
||
|
Appendix B: Instructions for Think Aloud Protocol |
|
||
|
Appendix C: Instrument to Assess Model Reliability |
|
||
|
Appendix D: Procedure for Finalization of Schema |
|
||
|
Appendix E: Raw Data from Structured Interviews Sorted by Emergent Categories |
|
||
|
Appendix F: Individual Learning Plan |
|
||
|
|
|
||
Many high school students, post-secondary students, and adults of working age have difficulty making career decisions. Dysfunctional thinking about themselves, their options, and their decision-making abilities have been shown to contribute to indecision, anxiety, and depression, which may in turn inhibit career development. While assessment instruments such as the Career Thoughts Inventory (CTI) and Workbook can be used by practitioners and clients to identify, challenge, and alter dysfunctional career statements no model exists for evaluating the quality of clients’ reframed thoughts, which result from such interventions. Such a schema may be helpful in determining the effectiveness of cognitive restructuring treatments like the CTI Workbook and other cognitive restructuring exercises. Furthermore, this schema may also be a useful tool in teaching practitioners and clients how to evaluate their thoughts and create more effective reframes. This report documents the creation and preliminary evaluation of such a model based upon the clinical judgments of a panel of “expert” career counselors.
A Model for Evaluating the Effectiveness of
Cognitive Reframes of Dysfunctional
Career Thoughts: Technical Report 36
Difficulty making career transitions is not uncommon for both adolescents and adults. For example, in interviews conducted with 1,000 youth transitioning from school to work, Mortimer, Zimmer-Gembeck, Holmes, and Shanahan (2002) identified themes of unfulfilled expectations and postponement of career-related decisions. Interview participants rarely referred to guidance practitioners as positive and constructive influences on their decision making. Similarly, “…many mid-life career changers must deal with the frustrations of having their core issues glossed over or even ignored…” by job search professionals (Behrens & Altman, 2000, p. 44). During their three year qualitative study, Behrens and Altman found that 95% of 92 mid-life career changers professed a lack of confidence in their ability to find mid-life career satisfaction and felt that their desire to explore such core issues was rebuffed by job search experts. The authors recommended that, “By helping them to conquer their fears, correct erroneous beliefs, and become aware of and revise negative self-talk…,” practitioners can help mid-life job seekers to “…find the energy to network and project the confidence to interview successfully.”
These career decision-makers, regardless of age, believe their needs are being ignored by practitioners and other career development professionals. This may in part be due to practitioners perceiving themselves as unprepared to help clients address the core issue of dysfunctional cognitions that complicate career choices. For example, at one college career center both anecdotal and qualitative data have been collected on the difficulties experienced by student practitioners using an assessment and intervention for dysfunctional career thinking with clients (Etheridge, in progress). It is not surprising that expert practitioners have been shown to have more complex schemata than those of novices for intervening with clients (Martin, Slemon, Hiebert, Hallberg, & Cummings, 1989). However, it is somewhat surprising that experts may have difficulty defining and communicating their sophisticated yet tacit schemata to practitioner trainees and clients.
One possible method of closing this gap in expertise between practitioners, practitioner-trainees, and clients, may be to develop an “expert model” for evaluating the effectiveness of client reframes of dysfunctional career related cognitions. In an attempt to offer such an expert model, this paper will briefly review the relevant theory base, describe a combined qualitative/ quantitative methodology and its derived results, and discuss these findings with respect to the empirical literature. This paper will then close with a discussion of the implications of the derived expert model for the delivery of career services, practitioner training, and additional research.
Career Problem Solving, Indecisiveness, and Dysfunctional Career Thoughts
Two fundamental realities of career problem solving are that it can be a cognitively complex and affectively unsettling task. Bruner (1985) pointed out that ordinary problems are often poorly defined; have multiple, interdependent goals; and shifting standards of success. Similarly, individuals attempting to resolve career problems are faced with ambiguous cues, interdependent alternatives, and uncertain outcomes. Furthermore, solving one career problem, usually creates only new problems requiring the client’s attention (Peterson, Sampson, & Reardon, 1991; Peterson, Sampson, Lenz, & Reardon, 2002; Sampson, Reardon, Peterson, & Lenz, 2004).
Because of this high level of cognitive complexity, many clients (like the previously mentioned transitioning adolescents and adults) have difficulty working through the career problem-solving process. However, it is the indecisive client who is generally thought to have a maladaptive approach to problem solving accompanied by a dysfunctional level of anxiety (Gordon, 1998; Peterson, Sampson, Jr., Reardon, & Lenz, 1996; Peterson et al., 1991; Sampson et al., 2004). The high anxiety of the indecisive client limits cognitive capacity and motivation for career choice. Intensifying emotion has been found to further bias recall and perception, thus creating a self-perpetuating cycle of increasing cognitive dysfunction and affect (Freeman, Pretzer, Fleming, & Simon, 1990). Such a reciprocal relationship between negative thinking and mood is supported by the available evidence in empirical studies by Teasdale and other investigators (Marzillier, 1986).
The conceptual evolution of career indecision has been described by Savickas (1995) as moving from a dichotomy, to a one-dimensional continuum, to a multidimensional concept. For example, in undergraduate college students, career indecision has been found to be a, “…complex problem space of both cognitive and affective variables that are in themselves highly interrelated and ostensibly interactive. Vocational identity, state and trait anxiety, locus of control, depression, and dysfunctional career thoughts are all significantly associated with the state of career indecision and with each other” (Saunders, Peterson, Sampson, & Reardon, 2000, p. 294). Specifically, Saunders found that dysfunctional career thoughts captured significant amounts of independent variation in career indecision (Saunders, 1998).
Cognitive Information Processing Theory: A Descriptive and Supportive Tool
In an effort to describe and intervene in the career choice process, numerous theories of career problem solving and decision making have been created. One theory which considers the effect of dysfunctional thoughts and accompanying negative affect on choice is the Cognitive Information Processing Approach to Problem Solving and Decision Making (CIP) (Peterson et al., 1996; Peterson et al., 1991; Peterson et al., 2002; Sampson et al., 2004). Similar to other theories of career development, CIP Theory describes the content and process of career choice. In this instance, the Pyramid of Information Processing Domains defines three content domains of knowledge, decision-making skills, and executive processing skills (Figure 1). The process component of CIP Theory is the five step CASVE cycle contained within the decision-making skills domain (Figure 2).
However, what is somewhat unique about CIP Theory is the Executive Processing Domain, which emphasizes the importance of metacognitive skills (Figure 1)(Peterson et al., 1991; Peterson et al., 2002; Reardon, Lenz, Sampson, & Peterson, 2000; Sampson et al., 2004). Assisting indecisive clients to become aware of their cognitive distortions requires that clients be aware of their metacognitions, that is to be able to “think about their thoughts.” The concept of metacognition has many different labels in the literature. These labels have included the executive processes of Belmont and Butterfield (1977) and what Gagne and Briggs (1974) called cognitive strategies. According to Wells (2000, p. 6), “…metacognition is defined as any knowledge or cognitive process that is involved in the appraisal, monitoring, or control of cognition.”
Several studies have found that young children possess limited metacognitive knowledge and ability (Flavell, 1979); thus, we must practice and develop metacognitive skills as we mature. Learning to use these metacognitive skills helps individuals “…to orchestrate cognitive aspects of problem solving” (Paris & Winograd, 1990, p. 18). Specifically, metacognitions can lead individuals to: 1) establish new goals and revise or abandon previous goals; 2) add to, delete from, or revise their metacognitive knowledge; or 3) activate strategies that generate additional cognitive or metacognitive goals (Flavell, 1979). When such metacognitive skills are limited, dysfunctional thoughts may be frequent and subsequent efforts to resolve the content or process problems of career choice are hindered.
CIP Theory outlines three kinds of overlapping metacognitive skills which can be learned by clients as a means to becoming more effective career decision makers: self-awareness, self-talk, and monitoring and control (Peterson et al., 1996; Peterson et al., 1991; Peterson et al., 2002; Reardon et al., 2000; Sampson et al., 2004). Such a delineation provides a framework to organize the theoretical contributions of Beck and others which both describe and offer possible interventions to the indecisive career problem solver.
“Good problem solvers are aware of their feelings as they engage in information-processing tasks” (Reardon et al., 2000, p. 88). Such feelings become one of the internal cues that prompt individuals to employ their metacognitive skills. External cues (e.g., messages from significant others and peers) and one’s cognitive and affective reactions to them are also important in the decision-making process. It is also important that the career decision maker be able to sort through these often conflicting internal and external cues to be able to identify the “real” problem or problems to be solved.
For example, a college student lacking in self-awareness may choose to major in accounting to please what she perceives as a demanding father, rather than fulfill her interest in the creative arts. What is the problem to be solved here? Is it her limited awareness of the degree to which she values her creative interests? Is it her inability to balance her own needs with those of her father’s? Could it be that her perception of the messages sent by dad is distorted by her continuing to play the role of dutiful daughter? The answers may be yes to all of these questions; however, awareness of one’s own feelings and thoughts is the key to finding them.
Awareness of one’s tacit assumptions and explicit thoughts is also critical to effective career decision making. Krumboltz (1983) noted that most clients are unaware of the biases, misinformation, and distorted beliefs that they bring to career choice and that these presuppositions can lead to self-defeating and disabling experiences as well as limit future learning opportunities (Krumboltz & Jackson, 1993). For example, Elliott (1995) noted that negative self-statements can impair a client’s ability to utilize occupational information, lead to career indecision, and result in inappropriate choices (Thompson, 1976; Lewis & Gilhousen, 1981; Nevo, 1987). This statement is supported by Bihm and Winer (1983) who found increasing thematic intrusions (recall errors) in recall of occupational information over time for those occupations with which subjects were already familiar. Our self-knowledge is thought to be similarly influenced by biases and distortions which color our recall and perception of past events (Sampson et al., 2004). The cognitive mechanism by which this phenomenon occurs will be explored next in an attempt to understand a second important metacognitive skill, management of self-talk.
The supposition that knowledge of the world of work and one’s self is integrated into a client’s preexisting and possibly biased schemata is supported by the previously discussed findings. Schemata are “internal models of aspects of self and the world that individuals use to perceive, store, or recall information” (Robins & Hayes, 1993, p. 205). In short, they provide a model of an individual’s cognitive organization and may be either conscious or tacit to the client (Marzillier, 1986; Mahoney, 1980). The schema concept was first discussed by Bartlett (1932), who realized the action of preexisting knowledge on the learning and recall of information (Mayer, 1983). Later, schemata were empirically found to resist modification as it is easier to assimilate information into a current schema than it is to change the schema itself (Thompson, 1976). This phenomenon, where feedback from the environment will result in adjustment always relative to the starting point, has been known as the adjustment from an anchor heuristic (Tversky & Kahneman, 1974), problem-solving set (Luchins, 1942), and by various other labels. Thus, divergent thought about a specific situation, object, option, or person is inhibited by past experience.
However, it is possible to assist clients in altering these schemata by helping them become more aware of the impact such schemata have on their automatic thoughts or self-talk (Beck, 1976). These thoughts are the easiest to access in therapy by both client and practitioner, are the least stable, and reflect the client’s perceptions of events rather than objective appraisals due to distortion and bias from the client’s schemata (Robins & Hayes, 1993). Thus, schemata can account for the genesis of negative thinking as well as its activation and maintenance (Marzillier, 1986). Since schemata also form the basis of a client’s beliefs and assumptions (which Beck & Weishaar, 2000, described as the most stable and least accessible form of cognition) a goal of the career practitioner may be to uncover automatic thoughts and deeper beliefs/assumptions, as reoccurrence of indecisiveness is more likely if both are not changed.
Cognitive distortions are the link between schemata and automatic thoughts (Robins & Hayes, 1993). For example, a client may only associate high earnings with business occupations and therefore may automatically dismiss the occupation of nurse when it is suggested by an interest inventory. These distortions are inaccurate, untestable rules that mediate client responses to environmental events and are often adhered to in the face of disputing evidence or after they cease to be functional (Zettle & Hayes, 1982). These distortions tend to overemphasize negative information to the exclusion of positive information (Hammen & Krantz, 1976).
Any given automatic thought may reflect more than one cognitive distortion (Robins & Hayes, 1993). Several specific distortions or systematic cognitive errors have been identified by Beck (Beck, 1976; Beck, 1967; Beck, Emery, & Greenberg, 1979) which are summarized in Appendix A. Arnkoff and Glass (1982) classified these and other cognitive distortions into two major categories, misuse of information (e.g., overgeneralization) and extreme types of thinking (e.g., dichotomous thinking). Thus, cognitive distortions are a reoccurring phenomenon in which an individual’s processing of information is biased in a classifiable and predictable manner. Limited self-awareness, dysfunctional thoughts based on cognitive distortions, and inaccurate schemata may inhibit clients from executing the third metacognitive skill necessary for career problem solving, monitoring and control.
Good problem solving and career decision making requires a “…thoughtful balance between compulsivity and impulsivity,” knowing when to stop the decision-making process and when to move ahead (Reardon et al., 2000, p. 88). Poor metacognitive skills in monitoring and control can yield behavioral extremes, from the compulsive decision maker on a never ending search for the “perfect” career to the impulsive decision maker whose trial and error attempts result in “trying out” several college majors (Peterson et al., 2002). The anxiety and feelings of frustration that result from the fallout associated with problems in monitoring and control may then only reinforce the client’s faulty schemata of themselves as poor career decision makers. Fortunately, there are interventions available that may help clients improve their metacognitive skills of self-awareness, self-talk, and monitoring and control in order to improve their underlying schemata.
A CIP-Based Intervention for Improving Metacognitions
Krumboltz (Krumboltz & Vidalakis, 2002; Krumboltz & Jackson, 1993; Krumboltz, Mitchell, & Jones, 1976; Krumboltz, 1976) noted the importance of using assessment not only for matching and evaluation purposes in career assessment, but also to promote learning in career decision making. One such assessment is the Career Thoughts Inventory and Workbook (CTI) (Sampson, Peterson, Lenz, Reardon, & Saunders, 1996a, 1996b). The CTI is a self-administered, objectively scored measure of dysfunctional cognitions involved in career problem solving and decision making. The 48 items of the CTI were derived via factor analysis from the best performing items of a pool of 248 items generated from the eight content dimensions of Peterson et al.’s CIP Theory (e.g., self-knowledge, option knowledge, metacognitive skills, and the five step CASVE cycle shown in Figure 1 and Figure 2), additional literature on dysfunctional career thinking, and actual client statements during service delivery (Sampson, et al., 1996c). The CTI has been standardized on high school, college student, and adult populations, and subsequently shown to have strong internal consistency, stability, and factorial, convergent, and predictive validity (Sampson, et al., 1996c).
The primary function of the CTI is to serve as a mechanism for screening and needs assessment (Sampson, Peterson, Lenz, Reardon, & Saunders, 1999). The CTI serves as a screening tool by assisting practitioners in making preliminary decisions about the levels of support they need to provide in relation to client needs (i.e., clients with higher levels of dysfunctional thinking are assumed to require greater levels of support). Furthermore, the CTI functions as a needs assessment tool by identifying the specific nature of a client’s dysfunctional career thinking through its factor-based construct scales and the analysis of individual item responses relative to CIP Theory’s eight content areas.
When combined with the companion booklet, Improving Your Career Thoughts: A Workbook for the Career Thoughts Inventory (workbook) (Sampson, Peterson, Lenz, Reardon, & Saunders, 1996a), the CTI is extended from a screening and needs assessment to a learning resource. The CTI Workbook is composed of five sections that are designed to help clients identify the amount and nature of their negative career thoughts, challenge and alter these thoughts, create a plan, and then take action.
First, a wall metaphor is used to interpret the instrument’s scales and describe the nature and impact of negative thinking on career decision making. Next, negative career thoughts are challenged by stimulus reframing statements which model effective thinking about career decision making for the client as recommended by Meichenbaum and Cameron (Meichenbaum, 1972; Meichenbaum & Cameron, 1974). Clients then alter their negative statements in the workbook to be more functional and they are encouraged to plan and take action based on these new statements. In addition to altering negative career thinking, workbook users are encouraged to make good use of social supports during decision making. In effect, the CTI Workbook emphasizes to the client the importance of the metacognitions of self-awareness, self-talk, and monitoring and control, by teaching the specific skill of cognitive reframing or altering negative self talk.
While the CTI assessment has been used in research since its release, the workbook has received little attention in the literature (Vernick, 1999; Sampson et al., 2004). Reviewers Gilbert (1996) and Fontaine (2001) noted the lack of evidence in the CTI professional manual of the workbook’s ability to correct dysfunctional career thinking or improve career problem-solving and decision-making skills. A search of the literature also revealed no formal, empirical evidence of its efficacy. This may be due in part to the fact that no model exists for evaluating the effectiveness of reframed career thoughts resulting from use of the workbook.
The design, participants, measures, and procedures employed to develop an expert model for evaluating the effectiveness of cognitive reframes of dysfunctional career thoughts will now be described.
The design of this
study combined both qualitative and quantitative methods. To create the expert
model of evaluating reframes of dysfunctional career thoughts, both structured
interview and “think aloud” protocol techniques were employed to provide
qualitative data of some depth. Such a depth interview is best used,
…when
the focus of inquiry is narrow, the respondents represent a clearly defined and
homogenous bounded unit with an already known context, the respondents are
familiar and comfortable with the interview as a means of communication and the
goal is to generate themes and narratives (Crabtree
& Miller, 1999, p. 90).
The
“think aloud” instructions employed by the investigator during the interview,
were intended to encourage experts to verbalize their thoughts as they
evaluated cognitive reframes. Originally employed by Newell and Simon (1972), such protocols have long been used to
identify the sequence of steps and knowledge used by experts when solving
discrete problems (e.g., playing a chess game). A second stage of the study,
relying on quantitative methods was also designed to tentatively establish the
inter-rater reliability of judgments among experts applying the model to
determine the degree to which a person has successfully reframed a
dysfunctional career thought. In short, the study was designed to use qualitative methods to make an
explicit model of the tacit schemata and rules used by experts in judging the
effectiveness of cognitive reframes, then quantitatively demonstrating that
this model can be applied in a reliable manner to judge a novel set of reframes
of dysfunctional career thoughts.
Participants
A purposeful sampling strategy was applied because
“experts” on the reframing of dysfunctional career thoughts were desired for
their specific knowledge about the topic and relative homogeneity in clinical
and theoretical approach (Kuzel, 1999). Although the ideal sample
size for qualitative research is from five to 20 units of analysis (Patton, 1990),
four doctoral level, counseling practitioners responsible for the development
of the CTI were selected for their perceived expertise in the use of the
instrument and the accompanying workbook. Keeping in mind that years of
experience has not been clearly shown to relate to expertise (Skovholt, Ronnestad,
& Jennings, 1997),
these four professionals possessed greater than 100 years combined experience
working with clients who have career and mental health problems. All four
experts were Caucasian, three male and one female, with ages ranging from 49 to
62 years (
= 55.5).
Individually, these professionals have written extensively on career development theory and practice, publishing numerous articles and books on various aspects of the subject. As a group, these practitioners have developed the Cognitive Information Processing Approach to Career Problem Solving and Decision Making (Peterson et al., 1991; Peterson et al., 1996; Peterson et al., 2002; Sampson et al., 2004). In short, the training, experience, and research credentials of the panel members qualified them as “expert” in the area of investigation. The participants consented to participate in this study after a brief description of the procedure and time requirements were provided by the investigator.
As this study was conducted in two stages, model creation and evaluation of model reliability, two instruments were also developed: 1) a structured interview to facilitate the development of a model based on experts’ schemata and 2) a rating form to determine the reliability of the new model when evaluating cognitive reframes. These two instruments are described in the following text.
Structured Interview. As part of the structured interview protocol, each of the four participants was presented with eight stimulus items drawn from the CTI (Sampson, Peterson, Lenz, Reardon, & Saunders, 1996b). Paired with each CTI item was a corresponding cognitive reframe selected from a preexisting dataset of 2,064 reframing statements generated as part of an experimental evaluation of the effectiveness of the CTI Workbook in an undergraduate career development course (Carr, 2003). Care was taken to ensure that the reframed statements were randomly selected while representing a balance of participants, treatment statuses, and data collection times so that reframes viewed by the expert panel would be representative of varying individuals and quality. Out of the sixteen statements selected, six of the statements came from participants in the control group, ten statements from treatment group participants, eight statements were collected immediately after treatment, and eight statements were sampled two weeks post treatment. Finally, each statement included in the interview protocol came from a different participant in the study.
To ensure both item overlap and variety among experts, two versions of the structured interview (Form A and Form B) were created. Both versions of the interview protocol form can be reviewed in Appendix B. The reader should note that grammar and spelling errors in reframes generated by participants were retained. While CTI items and paired reframes were varied across the forms, items corresponding across forms were matched by the eight information processing domains of CIP Theory (Sampson et al., 1996b). Thus, corresponding items and reframes on the structured interview forms were assumed to relate to theoretically similar knowledge domains (e.g., self-knowledge, executive processing, etc.).
Rating Form. The rating form to determine the reliability of the new model when evaluating cognitive reframes can be viewed in Appendix C. This 12 item form was created by choosing three statements from the CTI and then for each statement, selecting four reframes from the preexisting undergraduate data set. Using the schema model as a guide, reframes were purposively selected by the investigator to represent one of four positions on a Likert scale ranging from –1 (less effective) to 2 (more effective). This unbalanced scale was chosen to represent reframes that actually might be counterproductive (i.e., -1), allow for the possibility of a neutral or no change response (0), and acknowledge that some reframes may be partial (+1) or complete (+2). In this case, an unbalanced scale was called for given that higher precision was desired on one side of the scale than the other (Friedman & Amoo, 1999).
Model Creation. To create the model, data were collected by appointment, in the private offices of each of the four experts. Data collection times using the structured interview protocol ranged from approximately 30 minutes to 45 minutes in length. Members of the expert panel were given one of the two versions of the structured interview instrument (Appendix B) to follow throughout the procedure and were invited to write or draw on scratch paper at their discretion. Each expert was blind to the selected items and reframes until his or her interview was conducted and was asked not to discuss the structured interview with colleagues until data collection was complete.
Following the format of the structured interview, the expert was asked by the investigator to read each item and corresponding reframe aloud and then respond to prompts provided by the investigator. If after reading the reframe, the expert felt that the undergraduate participant who authored the reframe did not “understand the task,” then the remaining prompts were typically skipped and the investigator moved onto the next item-reframe combination. If after reading the reframe, the expert felt that the undergraduate participant did “understand the task,” additional prompts were delivered by the investigator. These prompts probed for the expert’s thoughts on the undergraduate’s understanding of the task, observations on how the undergraduate altered the statement, strategies used by the undergraduate, and the undergraduate’s degree of success and quality in reframing the statement. Experts were also asked to provide evidence supporting their decisions on reframe quality. Finally, experts were asked to share any additional thoughts they may have about the reframe. These self-reported thoughts were hand recorded with paper and pencil and then transcribed into a word processing program by the investigator.
After all structured interviews were conducted, statements collected from experts were combined, sorted into groups of related concepts, then summarized by the investigator using a method in keeping with the editing organizing style suggested by Addison (1999). Seven major categories or themes emerged from the data. Within each of these seven categories, “less effective” and “more effective reframes” of cognitive distortions were then described using statements made by the expert panel. The resulting draft model was presented to the panel for corroboration, comment, and revision; which resulted in minor changes in wording and formatting made to enhance clarity.
Model reliability. Next, experts were asked to rate a set of 12 reframes using the model and the rating form to assess its reliability (Appendix C). For scheduling reasons, this task was not performed under controlled conditions; however, panel members were asked to not communicate with each other during the rating process. The ratings of the four experts were then compared to each other and with those of the principal investigator in an attempt to determine the ability of the new model to reliably classify reframes on a dimension of global effectiveness.
Validity of Schema
Structure. The four member, expert
panel was again employed to assess the validity of the categories derived from
the criteria resulting from the think aloud protocols. Each member was given a
randomized a set of 25 cards, each containing one schema criterion, and was
asked to sort them into one of the seven category headings (e.g., locus of
control, explicitness, etc.). Members were
also provided with a “No Classification” category under which they could choose
to place those criteria, which were judged not to fit under any of the seven
categories.
After a review of
the initial sort results and at the suggestion of the expert panel, a second
round of categorization was conducted. The previous procedure was followed with
two changes: 1) a fifth expert, who was also an author of the Career Thoughts
Inventory and Workbook, was added to the panel to provide an odd number of
members to “break ties”; and 2) panel members were able to place each criterion
under both a primary and secondary category. Items on which agreement were not
achieved by at least 4 of the 5 expert members were reviewed by the original
four panel members using a structured format facilitated by the primary
investigator (Appendix D). The goal of this process was to arrive at
consensus on criterion categorization while ensuring that each member had an
equal chance to express a rationale for his or her prior judgments.
Schema Creation
Out of 32 potential judgments about reframe effectiveness,
experts decided 12 times that undergraduate students did not appear to
understand the reframing task. Thus, additional qualitative data was collected
on at least 20 expert judgments by the panel of four professionals (Table 1). Additionally, part of the expert judgment process
was to rate the general quality of each reframe on a scale from marginal (1) to
high (3) (Table 2). Of 32 potential judgments, five were rated as marginal
quality, 13 as moderate quality, and five as high quality. Quality ratings were
skipped for nine of the 12 judgments for which the task appeared to not be
understood. Thus, the majority of the reviewed reframes were judged by the
experts to have been the result of an understood task and were determined to
have been of mixed quality (i.e., the reframe data was valid and varied enough
to be judged by the experts).
The resulting raw data from the depth interview and think aloud process can be seen in Appendix E. The output of the qualitative sort procedure on this raw data was a Schema of Cognitive Distortions Influencing Career Problem Solving and Decision Making (Table 3). The seven themes or categories which emerged from the sort included: locus of control, explicitness, affect, time sensitivity, word choice, missing key points, and reframe structure.
Schema Reliability
Data resulting from the experts’ judgments for each CTI
statement and reframe after applying the model can be seen in Table
4, while average ratings for each expert can be viewed in Table
5. Given the range of the Likert scale and number of reframes rated for
each item, the expected mean score for each CTI statement would be .5 (i.e., -1
+ 0 +1 +2 = 2/4 =.5). It should be noted that the average ratings of two
experts agreed with those of the investigator, though this may be due to a
cuing effect caused by the design of the instrument. However, Dr. B appeared to
be more conservative and Dr. C appeared to be more liberal in their judgments
respectively ( =.25 and =.75).
There was also a moderate to high degree of correlation of reframe ratings with four of six comparisons among the four raters significant (α ≤ .05) (Table 6). Also, the judgments of three of the four experts correlated significantly with the a priori judgments of the investigator (α = .01). However, it appears that the judgments of Dr. C, were significantly correlated only to those of Dr. D. Furthermore, inter-rater agreements varied with Kappa coefficients ranging from .11 to .52 among experts and from -.07 to .77 between experts and the investigator (Cohen, 1960) (Table 7).
Validity of Schema Structure
The degree of
agreement among experts for the schema structure validation procedure can be
viewed in Table 8. Agreement, defined as four of the five
experts concurring on category placement, was achieved for 19 of the 25
criteria. Agreement was not initially achieved on six of the 25 criteria. A
group consensus about categorization was reached on five of these six criteria
during the structured discussion led by the principal investigator. The
remaining criterion on which agreement could not be achieved was removed from
the schema based on the unanimous recommendation of the expert panel.
Overall, four of
principal investigators original categorizations were changed by the expert
panel (Table 9). Three criteria were moved from other
categories into the category of affect and one criterion was moved to the
category of missing key point. The final table, with validated category groupings
can be viewed in Table 10. The close reader will notice
several criteria were slightly edited based upon the recommendation of the
expert panel. This action was taken to clarify meaning and intended rationale
for category membership to the reader.
This discussion will begin with an acknowledgement of the study’s limitations, discuss the literature in support of the constructed model, and close with suggestions for applying the model in practice and research.
A reader familiar with empirical studies might suggest that the structured interviews could have been conducted by a third party naïve to the career development and cognitive reframing literature. Furthermore, the evaluative statements made by clinicians in the first phase of the project could have been sorted by more than one similarly naïve individual. Instead data were collected and then sorted by the investigator. While steps could have been taken to reduce the risk of investigator bias in the creation of the model, such efforts may have hindered the collection of data (e.g., recorder’s lack of familiarity with terminology and nuances in expert statements) and resulted in a model with less theoretical validity and clinical utility. Such practices are not uncommon in qualitative research, though the execution of the data collection and sorting process could have been more detailed and rigorous if individuals other than the investigator collected and sorted the data (Addison, 1999).
It should be noted that the volunteer experts were highly motivated to participate in this study. Furthermore, their common interests in theory and practice issues related to the role of dysfunctional thinking in career decision making via CIP Theory may limit the “generalizability” of the constructed model. Thus, the judgments of these expert practitioners may not be representative of other professionals with similar clinical and research experiences, thus the model may not be easily transferable to other settings.
Finally, the poor reliability of ratings made by experts on novel reframes with the model must be addressed. First, the small number of ratings (n = 48) made by the experts, left little room for error. However, it should be noted that the formatting of the reframe rating instrument may have suggested to raters that each point on the four point rating scale corresponded to one of the four reframes in the CTI item grouping. Thus, a process of elimination may have also impacted rating independence. It would have been better to randomly present each CTI item and reframe pair to the experts.
Evidence supporting the validity of the proposed model exists in the theoretical and empirical literature. Most likely the expert panel’s collective knowledge of this literature is reflected in the model. Each of the seven themes contained in the model, (locus of control, missing key points, explicitness, affect, time sensitivity, word choice, and reframe structure) will now be discussed in the context of the problem-solving and metacognitive literature.
Locus of Control. Locus of control (LOC) is the degree to which individuals expect that reinforcement is contingent on their own behaviors versus environmental events resulting from chance or others’ actions. This construct has been found to be a relatively stable, cross situational measure of individual difference in numerous studies (Rotter, 1990). Specifically, LOC has been found to be a significant variable in the performance of problem-solving tasks (Larson, Piersel, Imao, & Allen, 1990). That is, if individuals think (tacitly or explicitly) that feedback necessary for success is dependent on external factors beyond their control, they are less likely to successfully solve their problems.
In the specific case of career decision making, LOC has
been found to be modestly associated with dysfunctional career thinking (r = .26, p 
.001) as measured by
the Career Thoughts Inventory (Saunders et al., 2000). However, while dysfunctional
thinking has been found to capture a significant amount of the variance on a
measure of career indecision, LOC has not. Thus, an external locus of control
may be a “symptom” of the dysfunctional thinking which interferes with the
problem-solving process (Peterson et al., 1991;
Peterson et al., 1996).
For example, Sue, who has been unsuccessfully searching for a job, has an excellent resume and cover letter. However, she does not follow up with employers after submitting her application for vacant positions. Instead, she expects that employers “will get in touch with her” if they like her resume (i.e., she waits for feedback from her environment). This may be due to a dysfunctional thought that she would be viewed as being “pushy” if she made a phone call politely expressing her continued interest.
Missing Key Point. In several reframes, the expert panel observed that what was negative or unhelpful about the original dysfunctional thought was not properly identified, thus the reframe “missed the point.” From the experts’ point of view, the client incorrectly defined the problem. However, there is evidence that clients can be trained to better identify and define problems to be solved. For example, in a study by Cormier, Otani, and Cormier (1986), highly specific training in problem definition was provided in which subjects were taught to employ a relatively simple mnemonic, ABC (Affect, Behavior, and Cognition) to describe the problem and identify its probable causes. This and other studies on the effectiveness of various problem-solving training protocols have suggested that the learning of problem definition skills may contribute to subjects choosing “better options” from among a variety of alternatives (Nezu & D'Zurilla, 1981a; Nezu & D'Zurilla, 1981b; Cormier, Otani, & Cormier, 1986).
For example, after trying out and disliking his third college major, Bob stated to his counselor, “I’ve tried all the good majors (Accounting, Purchasing, and Marketing) and I’m bad at them.” Bob views the problem as his “badness” (i.e., lack of skill) at “good majors” (i.e., business majors). After listening to Bob and considering the Pyramid of Information Processing Domains in Career Decision Making (Figure 1) (Sampson et al., 2004), the counselor decides that dysfunctional thoughts about choosing a major are limiting Bob’s knowledge of both himself and his options. Using the pyramid, the counselor helps Bob redefine the problem gap from one that is unbridgeable (i.e, “I’m bad”) to one that is passable, by engaging in learning experiences to improve his schemata of self and options and reframe his dysfunctional cognitions.
Explicitness.
This theme suggests a rational and concrete (i.e., observable) approach to
career decision making, which is in agreement with the primary role of cognitive therapy, “…to identify, reality test, and
correct distorted conceptualizations and the dysfunctional beliefs (schemata)
underlying these cognitions (Beck et al., 1979). In the spirit of Kelly’s (1955) person as scientist metaphor, the
cognitive-behavioral practitioner encourages clients to question and search for
observable evidence to support their dysfunctional thoughts and distortions.
Thus, one goal of
the practitioner may be to help clients learn to create explicit, “planful”
reframes, that are based on observable data. Heppner and Krauskopf (1987) emphasized the importance of helping
clients learn to state their problems and goals concretely in counseling. For
example, specific criteria for evaluating consequences of problem-solving
alternatives increased the chances of the “most effective” solution being
selected when compared to control (Nezu & D'Zurilla, 1979). Such specific self-talk may help clients
to exercise more effective control and monitoring, because they will have
criteria by which they can judge if they have successfully achieved their goal.
For example, a
guidance counselor working with Tonya, a high school senior interested in
nursing may hear her say, “All nurses go to four year colleges, I can’t handle
that much school.” The counselor first helps Tonya identify the
cognitive distortions reflected in her thought. Next, he helps her challenge
her dysfunctional thought by asking her to detail the evidence supporting her
claim that “all nurses require 4-year degrees.” When she realizes that she
lacks evidence supporting her assumption, the counselor asks Tonya to review
several nursing occupations and their education requirements on a
computer-assisted career guidance system. Once enough evidence is collected,
the counselor helps Tonya alter her schema and thoughts about nursing
occupations. Finally, Tonya is encouraged to act on her new found knowledge
by interviewing admissions counselors at both 2- and 4-year nursing programs.
In turn, the new evidence Tonya collects can be used to challenge and alter her
statement that she, “can’t handle that much school.” Tonya soon learns that
instead of the zero options afforded by her original thought, she actually has
several options from which to choose.
Affect. A significant misunderstanding about cognitive approaches is that they ignore affect (Freeman et al., 1990; Sampson et al., 2004). This model, in keeping with the previous discussion of Cognitive Information Processing theory, helps to refute this myth by stressing the importance of addressing affect and strategies for emotional coping in cognitive reframes where appropriate. While cognitive practitioners usually choose to intervene with cognitions first, as they are seen as the mediator between environmental stimuli, feeling, and behavior, it may be important for cognitive reframes to acknowledge the affective factors influencing career decision making. For example, the induction of an affectively positive state has been found to improve performance on problem-solving tasks (Isen, Daubman, & Nowicki, 1987). Marzillier (1986, p. 100) concluded that debate about the causal relationship of cognitions to emotions by stating that, “…a dynamic interdependence between thought, feeling, and action describes their relationship more usefully than any assertion of primacy.”
The impact of affect on career decision making is illustrated by the case of Louisa, a skilled worker downsized from a local electronics manufacturer. After job hunting for three months, she visited her local workforce center, just as her unemployment was about to expire. She told her counselor, “I’m at the end of my rope, even if I could afford food I couldn’t eat, I’m so upset…all I do is cry. I’ve gone on 10 interviews and they all hated me, I’m unemployable.” It’s obvious that Louisa is depressed. Her cognitive distortions (e.g., they all hated me) bias her view of herself and her depressive affect (e.g., tearfulness and brooding) bias employers’ views of Louisa. The counselor decided to refer Louisa to a medical professional for further evaluation and possible pharmacological treatment of her depression. Once the affective symptoms of depression lift; reframing work on dysfunctional thoughts such as “I’m unemployable” can progress by evaluating the evidence derived from her past ten interview experiences. The evidence may show that Louisa simply needs to improve her interviewing skills.
Time Sensitivity. How the client perceives time and its importance in problem solving may also play an important role in successfully reframing dysfunctional cognitions. For example, the setting of interim or proximal goals (e.g., breaking long periods of time into chunks) may be helpful to clients. “When individuals are somewhat unsure of their ability to handle a complex, challenging endeavor, the setting of short-term, proximal sub goals can beneficially affect self-referent thought, motivation and performance” (Stock & Cervone, 1990, p. 496). Furthermore, Stock and Cervone noted that when setting interim goals that flexibility be maintained so as to not diminish a client’s sense of control and motivation.
Practitioners may also encourage clients who are reframing cognitions to consider the length of time and amount of effort required to arrive at a career decision. For example, Sandelands, Brockner, and Glynn (1988) found that graduate students exhibited greater persistence in solving insoluble tasks when they were informed that persistence was a wise strategy. Similarly, helping clients to acknowledge the ongoing nature of decision making (i.e., solving one problem leads to new problems to be solved) may induce a similar problem-solving set that encourages success (Reardon et al., 2000). Thus, initially helping clients to clarify the time and persistence required for problem solving may improve problem-solving performance.
An Individual Learning Plan (ILP) (Peterson et al., 1996; Sampson et al., 2004) can be used in career counseling by practitioners to clarify a client’s goals and her perception of the time and persistence required. This form, viewable in Appendix F, prompts the client to state and record her distal goals. Also included on the ILP are learning activities, collaboratively generated by both client and counselor, to facilitate goal attainment. The intended purpose of (i.e., proximal goal) and time required to complete each activity is also written on the ILP. The clarity provided by the ILP may provide a sense of, “buy in for the client, increasing the likelihood that the client will follow through”(Sampson et al., 2004, p. 60). Finally, consistent with the literature, it is clear to the client at the bottom of the ILP that goals can be changed as learning occurs.
Word Choice. Perhaps one of the most obvious themes to emerge from the model is that of word choice. As discussed previously in this paper, Beck and others (Beck, 1976; Beck, 1967; Beck et al., 1979) have identified several cognitive distortions present in the self-talk of clients which have been categorized into misuse of information and extreme types of thinking (Arnkoff & Glass, 1982) (Appendix A). Among these distortions, overgeneralization, magnification and minimization, dichotomous thinking, and imperative statements are perhaps the easiest for the novice practitioner and client to identify due to the specific words associated with them (e.g., always, all, none, never, must).
However, one of the most difficult aspects for clients to learn may be the difference between “not negative self talk” and “positive self-talk.” Judgments by the expert panel indicated that simple word reversal (e.g., I can’t make a career decision to I will make a career decision) is not sufficient to achieve reframing. In fact, setting a problem-solving goal without properly defining the problem and creating an internally centered, explicit plan that is sensitive to issues of affect and time may be counterproductive. A more helpful reframe for the previous example might be, “While I might get anxious, I can stay calm and make a career decision if I follow the Individual Learning Plan I created one step at a time.” While this reframe may be somewhat complex, simpler statements may result from clients “moving through the motions” of the reframing process and in turn setting themselves up for additional failure and frustration.
Reframe Structure. The final theme to emerge in the model is the view that the structure of the reframe should parallel the structure of the original negative thought. In this case, the expert panel may not be attending solely to explicit verbal syntax, but also to the relationships it represents (e.g., If-then, action and consequence). For example, when asked to categorize physics problems, novices used surface features of the problems such as equipment employed while experts used the theories needed to solve the problem (Chi, Feltovich, & Glaser, 1981). Chi suggested that this deep processing by experts is a result of their more sophisticated schemata.
The novice practitioner may hear her client’s statement “I’m afraid if I try out my chosen occupation, I won’t be successful” as a simple statement of fact. The expert, however, may derive deeper meanings about her client’s avoidance behavior resulting from anxious feelings and an external locus of control. By encouraging the client to maintain the implicit “If-Then” structure and reframe the statement in more positive and explicit terms, the expert practitioner may promote the acquisition of new knowledge; that anxiety and avoidance behavior decrease when one takes positive action.
Implications for Practice and Research
While some may see irony in the dichotomous format of the model’s presentation in Table 10 for describing dysfunctional career thoughts, its parsimony and utility are thought to outweigh its shortcomings. Hopefully, this model will enable practitioners of various skill and experience levels to make global judgments about the effectiveness of a given client reframe of a dysfunctional career thought. Once this global judgment is made, the clinician could then investigate the nuances of a possibly dysfunctional thought. In addition, this model is not intended for use as a checklist comparing the total number of less effective qualities to more effective qualities for a given reframe. It is intended as a guide to obtaining a sense of the overall quality of reframes of dysfunctional career thoughts.
The goal of cognitive-behavioral practitioners is to teach their clients to “autorestructure” their dysfunctional thoughts (i.e., become independent of the practitioner). However, how do clients know that their newly reframed thoughts are helpful? Sampson, Peterson, Lenz, and Reardon (1992) have provided clients with “plain language version” of the pyramid of information processing and CASVE cycle in an effort to ensure the utility of CIP Theory. Similarly, it may be equally helpful to provide clients with a job aid to evaluate the quality of their reframes. Such a client aid is available in Table 11 and is shown tied to the original expert model in Table 12.The reading level of this “plain language’ document has been found to be at the 7th grade level due to selected words, such as “negative” (Harris & Jacobson, 1982).
It should be noted that the limited reliability results were accomplished without training raters on the model in order to achieve higher levels of agreement. While it might be counterintuitive to expect “experts” to require training on rating data using a model based on their own schemata, one must separate the task of “rating” items for purposes of comparison from the task of applying the model ipsatively. Regardless of its suitability for use in research, a short training program might be created so practitioners can learn to use the model to assess and improve the quality of client reframes. If designed properly, an additional benefit of such training may be more reliable rating data that can be used to evaluate the outcomes of interventions for dysfunctional career thinking such as the CTI Workbook.
Evidence exists both in the literature and anecdotally that adolescents and adults are sometimes unsatisfied with career counseling that misses their “core issues.” From the general perspective of cognitive-behavioral therapy, these core issues may arise from the client’s obsolete schemata, cognitive distortions, and dysfunctional thoughts. From the specific perspective of Cognitive Information Processing theory, dysfunctional thoughts can contribute to career indecisiveness by undermining current, and preventing the learning of new, metacognitive skills necessary for successful career problem solving and decision making. This paper has offered a preliminary expert model which both practitioners and clients might use to create and evaluate more effective reframes of dysfunctional career thoughts. In turn, this model will hopefully enhance practice and research, which will result in greater satisfaction and success for both practitioners and clients.
Addison, R. B. (1999). A Grounded Hermeneutic Editing
Approach. In B. F. Crabtree & W. L. Miller (Eds.), Doing qualitative research (2nd ed., pp. 145-161).
Arnkoff, D. B., & Glass, C. R. (1982). Clinical cognitive constructs: Examination, evaluation, and elaboration. In P.C.Kendall (Ed.), Advances in cognitive-behavioral research (pp. 1-34). Academic Press.
Bartlett, F. (1932). Remembering:
A study in experimental and social psychology.
Beck, A. T. (1967). Depression:
Clinical, experimental, and theoretical aspects.
Beck, A. T., Emery, G., & Greenberg, R. L. (1979). Cognitive therapy of depression.
Beck, A. T., & Weishaar, M. E. (2000). Cognitive
therapy. In R.J.Corsini & D. Wedding (Eds.), Current psychotherapies (6th ed., pp. 241-272).
Beck, A. T. (1976). Cognitive
therapy and the emotional disorders.
Behrens, T., & Altman, B. (2000). Counseling mid-life career changers. Journal of Career Planning and Employment, 60, 44-48.
Belmont, J. M., & Butterfield, E. C. (1977). The instructional approach to the developmental cognitive research. In R.V.Kail & J.W.Hagen (Eds.), Perspectives on the development of memory and cognition (pp. 437-481). Hillsdale, N. J.: Erlbaum.
Bihm, E. M., & Winer, J. L. (1983). The distortion of memory for careers: The influence of the thematic organization of occupational information. Journal of Vocational Behavior, 23, 356-366.
Bruner, J. (1985). On teaching thinking: An afterthought. In S.F.Chipman, J. W. Segal, & R. Glaser (Eds.), Thinking and learning skills (pp. 407-426).
Carr, D. (2003). The
effect of a workbook intervention on college student's reframes of negative
career thoughts (Rep. No. 37).
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121-152.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
Cormier, W. H., Otani, A., & Cormier, S. (1986). The effects of problem-solving training on two problem-solving tasks. Cognitive Therapy and Research, 10, 95-108.
Crabtree, B. F., & Miller, W. L. (1999). Depth
interviewing. In B.F.Crabtree & W.L.Miller (Eds.), Doing qualitative research (2nd ed., pp. 89-107).
Elliott, K. J. (1995). Anthetic dialogue: A new method for working with dysfunctional beliefs in career counseling. Journal of Career Development, 22, 141-148.
Etheridge, R. (2003). The
construction of a test use competency measure in career counseling: A Piagetian
framework. The
Flavell, J. H. (1979). Metacognition and cognitive monitoring. American Psychologist, 34, 906-911.
Fontaine, J. H. (2001). Career Thoughts Inventory. In
B.S.Plake & J.C.Impara (Eds.), The
fourteenth mental measurements yearbook (pp. 228-230).
Freeman, A., Pretzer, J., Fleming, B., & Simon, K. M.
(1990). Clinical applications of
cognitive therapy.
Friedman, H. H., & Amoo, T. (1999). Rating the rating scales. Journal of Marketing Management, 9, 114-123.
Gagne, R., & Briggs, L. (1974). Principles of instructional design.
Gilbert, H. B. (1996). Career Thoughts Inventory: A review and critique (Rep. No. ED408526).
Gordon, V. N. (1998). Career decidedness types: A literature review. The Career Development Quarterly, 48, 386-402.
Hammen, C. L., & Krantz, S. (1976). Effect of success and failure on depressive cognitions. Journal of Abnormal Psychology, 85, 577-586.
Harris, A. J., & Jacobson, M. D. (1982). Basic
Heppner, P. P., & Krauskopf, C. J. (1987). An information-processing approach to personal problem solving. The Counseling Psychologist, 15, 371-447.
Isen, A. M., Daubman, K. A., & Nowicki, G. P. (1987). Positive affect facilitates creative problem solving. Journal of Personality and Social Psychology, 52, 1122-1131.
Kelly, G. (1955). The
psychology of personal constructs.
Krumboltz, J. D. (1983). Private
rules in career decision making (Special Publications Series No. 38) (Rep.
No. ED 229 608).
Krumboltz, J. D., & Vidalakis, N. K. (2002). Expanding learning opportunities using career assessments. Journal of Career Assessment, 8, 315-327.
Krumboltz, J. D. (1976). This Chevrolet can't float or fly. The Counseling Psychologist, 76, 17-19.
Krumboltz, J. D., & Jackson, M. A. (1993). Career assessment as a learning tool. Journal of Career Assessment, 1, 393-409.
Krumboltz, J. D., Mitchell, A., & Jones, G. (1976). A social learning theory of career selection. The Counseling Psychologist, 76, 71-80.
Kuzel, A. J. (1999). Sampling in qualitative inquiry. In B.
F. Crabtree & W. L. Miller (Eds.), Doing
qualitative research (2nd ed., pp. 33-46).
Larson, L. M., Piersel, W. C., Imao, A. K., & Allen, S. J. (1990). Significant predictors of problem-solving appraisal. Journal of Counseling Psychology, 37, 482-490.
Lewis, R. A., & Gilhousen, M. R. (1981). Myths of career development: A cognitive approach to vocational counseling. The Personnel and Guidance Journal, 59, 296-299.
Luchins, A. S. (1942). Mechanization in problem solving -- the effect of Einstellung. Psychological Monographs, 54, 95.
Mahoney, M. J. (1980). Psychotherapy and the structure of
personal revolutions. In M.J.Mahoney (Ed.), Psychotherapy
process: Current issues and future directions (pp. 157-180).
Martin, J., Slemon, A. G., Hiebert, B., Hallberg, E. T., & Cummings, A. L. (1989). Conceptualizations of novice and experienced counselors. Journal of Counseling Psychology, 36, 395-400.
Marzillier, J. S. (1986). Changes in depressive beliefs: An
analysis of Beck's cognitive therapy for depression. In P.C.Kendall (Ed.), Advances in cognitive-behavioral research
and therapy: Vol. 5. (pp. 89-114).
Mayer, R. E. (1983). Thinking,
problem solving, cognition.
Meichenbaum, D., & Cameron, R. (1974). The clinical potential of modifying what clients say to themselves. Psychotherapy: Theory, Research, and Practice, 11, 103-117.
Meichenbaum, D. (1972). Ways of modifying what clients says
to themselves.
Mortimer, J. T., Zimmer-Gembeck, M. J., Holmes, M., & Shanahan, M. J. (2002). The process of occupational decision making: Patterns during the transition to adulthood. Journal of Vocational Behavior, 61, 439-465.
Nevo, O. (1987). Irrational expectations in career counseling and their confronting arguments. The Career Development Quarterly, 35, 239-250.
Newell, A., & Simon, H. A. (1972). Human problem solving.
Nezu, A., & D'Zurilla, T. J. (1979). An experimental evaluaton of the decision-making process in social problem solving. Cognitive Therapy and Research, 3, 269-277.
Nezu, A., & D'Zurilla, T. J. (1981a). Effects of problem definition and formulation on the generation of alternatives in the problem-solving process. Cognitive Therapy and Research, 5, 265-271.
Nezu, A., & D'Zurilla, T. J. (1981b). Effects of problem defintion and formulation on decision-making in the social problem-solving process. Behavior Therapy, 12, 100-106.
Paris, S. G., & Winograd, P. (1990). How metacognition
can promote academic learning and instruction. In Beau Fly Jones & Lornal
Idol (Eds.), Dimensions of Thinking and
Cognitive Instruction (pp. 15-51).
Patton, M. Q. (1990). Qualitative
evaluation and research methods (2nd ed.).
Peterson, G. W., Sampson, J. P., Jr., Reardon, R. C., &
Lenz, J. G. (1996). A cognitive information processing approach to career
problem solving and decision making. In D.Brown & L. Brooks (Eds.), Career choice and development (3rd ed.,
pp. 423-475).
Peterson, G. W., Sampson, J. P. Jr., Lenz, J. G., &
Reardon, R. C. (2002). A cognitive information processing approach to career problem
solving and decision making. In D.Brown (Ed.), Career choice and development (4th ed., pp. 312-369).
Peterson, G. W., Sampson, J. P. Jr., & Reardon, R. C.
(1991). Career development and services:
A cognitive approach.
Reardon, R. C., Lenz, J. G., Sampson, J. P., & Peterson,
G. W. (2000). Career planning and
development: A comprehensive approach.
Robins, C. J., & Hayes, A. M. (1993). An appraisal of cognitive therapy. Journal of Consulting and Clinical Psychology, 61, 205-214.
Rotter, J. B. (1990). Internal versus external control of reinforcement: A case history of a variable. American Psychologist, 45, 489-493.
Sampson, J. P., Jr., Peterson, G. W., Lenz, J. G., & Reardon, R. C. (1992). A cognitive approach to career services: Translating concepts into practice. The Career Development Quarterly, 41, 67-74.
Sampson, J. P., Jr., Peterson, G. W., Lenz, J. G., Reardon,
R. C., & Saunders, D. E. (1996a). Career
Thoughts Inventory.
Sampson, J. P., Jr., Peterson, G. W., Lenz, J. G., Reardon,
R. C., & Saunders, D. E. (1996b). Improving
your career thoughts: A workbook for the Career Thoughts Inventory.
Sampson, J. P. Jr., Peterson, G. W., Lenz, J. G., Reardon,
R. C., & Saunders, D. E. (1996c). Career
Thoughts Inventory Professional Manual.
Sampson, J. P. Jr., Peterson, G. W., Lenz, J. G., Reardon, R. C., & Saunders, D. E. (1999). The use and development of the Career Thoughts Inventory (Rep. No. ED447362).
Sampson, J. P. Jr., Reardon, R. C., Peterson, G. W., &
Lenz, J. G. (2004). Career counseling and
services: A cognitive information processing approach.
Sandelands, L. E., Brockner, J., & Glynn, M. A. (1988). If at first you don't succeed, try, try again: Effects of persistence-performance contingencies, ego involvement, and self-esteem on task persistence. Journal of Applied Psychology, 73, 208-216.
Saunders, D. E. (1998). The contribution of depression and dysfunctional career thinking to career indecision. Dissertation Abstracts International, 58 (7-B), 3953.
Saunders, D. E., Peterson, G. W., Sampson, J. P. J., & Reardon, R. C. (2000). Relation of depression and dysfunctional career thinking to career indecision. Journal of Vocational Behavior, 56, 288-298.
Savickas,
M. L. (1995). Current theoretical issues in vocational psychology: Convergence,
divergence, and schism. In W. B. Walsh & S. H. Osipow (Eds.), Handbook of vocational psychology: Theory,
research, and practice (2nd ed.): Contemporary topics in vocational psychology
(pp. 1-34).
Skovholt, T. M., Ronnestad, M. H., &
Stock, J., & Cervone, D. (1990). Proximal goal-setting and self-regulatory processes. Cognitive Therapy and Research, 14, 483-498.
Thompson, A. P. (1976). Client misconceptions in vocational counseling. The Personnel and Guidance Journal, 55, 30-33.
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124-1131.
Vernick, S. H. (1999). The Career Thoughts Inventory (CTI) in research and practice (Rep. No. ED445304).
Wells, A. (2000). Emotional
disorders and metacognition: Innovative cognitive therapy.
Zettle, R. D., & Hayes, S. C. (1982). Rule governed behavior: Potential theoretical framework for cognitive behavioral therapy. In P.C.Kendall (Ed.), Advances in cognitive-behavioral research (pp. 73-118). Academic Press.
|
Expert Judgment
of Task Understanding |
||||
|
|
|
|||
|
|
CTI Items 33-40 |
CTI Items 41-48 |
||
|
item |
Dr. A |
Dr. B |
Dr. C |
Dr. D |
|
1 |
Yes |
Yes |
Yes |
Yes |
|
2 |
Yes |
Yes |
Yes |
Yes |
|
3 |
Yes |
Yes |
somewhat |
Yes |
|
4 |
Yes |
No |
No |
No |
|
5 |
Yes |
No |
somewhat |
Yes |
|
6 |
No |
No |
No |
No |
|
7 |
Yes |
Yes |
No |
No |
|
8 |
No |
No |
somewhat |
Yes |
|
Ratings of
Reframe Quality by Experts |
||||
|
|
||||
|
|
CTI Items 33-40 |
CTI Items 41-48 |
||
|
item |
Dr. A |
Dr. B |
Dr. C |
Dr. D |
|
1 |
3 |
2 |
3 |
3 |
|
2 |
1 |
2 |
3 |
3 |
|
3 |
2 |
2 |
2 |
2 |
|
4 |
2 |
1 |
skipped |
skipped |
|
5 |
2 |
1 |
2 |
2 |
|
6 |
skipped |
skipped |
skipped |
skipped |
|
7 |
2 |
2 |
skipped |
skipped |
|
8 |
skipped |
1 |
2 |
1 |
|
|
|
|
|
|
|
Legend: 1=
marginal, 2=moderate, 3=high |
||||