CHAPTER 18 Toward Instructional Process Measurability: An Interbehavioral Field Systems Perspective

Andrew Hawkins

Tom Sharpe

Roger Ray

We are seeing a change in focus from the study of one organism over time to the study of the social interaction between organisms.

Dillon, Madden, and Kumar (1983, p. 564)

It has been an ongoing challenge for educational researchers to systematically capture the many observable variables relevant to instruction as they occur in concert in natural settings. At a conceptual level, many similarities exist between an interbehavioral field systems perspective and the nature of the instructional milieu. The purpose of this chapter is to take advantage of these similarities by describing the value of the field systems orientation as it relates to the measurement of instructional settings, and by detailing the feasibility of a measurement technology based on an interbehavioral framework. Specifically, we address this purpose by (1) conceptualizing a field systems perspective as it relates to the assessment of instructional processes, (2) describing an interbehavioral measurement system that uses current computer-based technology, (3) presenting an example of instructional assessment using a field systems perspective in which longitudinal data point to the efficacy of such an approach in the education and assessment of teachers, and (4) providing future potential applications of this approach specifically to teacher education and generally to behavior analysis. We hope this type of information will provide behavior analysts with the ability and incentive to focus on the more complex and observable process interactions that are naturally exhibited between teachers and students in context. Our goal is to contribute to the further development of data-driven, behavior-analytic assessment of instructional settings.

Conceptual Foundations

It is easy to agree with Jackson's (1968) perspective that in the context of typical classroom settings, events come and go with astonishing rapidity. Individual teachers engage in hundreds of behavioral interactions within each class period. Accurate observation and analysis is difficult at best and is at least partly responsible for a "science versus art" debate with regard to effective instruction. Many educators contend that a true science of teaching is beyond our grasp, for it "implies that good teaching will someday be attainable by closely following rigorous laws that yield high predictability and control" (Gage, 1978, p. 17). Some argue that effective teaching is spontaneous and intuitive. Clinical assessments within a class setting regarding how multiple variables affect the solution to the myriad of daily instructional problems are said to rely primarily on feeling and artistry (cf., Dawe, 1984; Eisner, 1983; Gage, 1984; Rubin, 1985). In essence, many mainstream contemporary educational researchers agree that scientific attempts to analyze teaching as a preplanned or predictive process are much too difficult and time-consuming. Further, they may even distract from a study of important esthetic, intuitive, and political factors.

Interbehavioral Field Systems

Behavior-analytic research in natural classroom settings has largely stemmed from a linear approach to experimentation and evaluation (Witt, 1990). Although productive, a linear perspective allows primarily for possible solutions to an immediate concern by examining a limited number of variables. Results are temporally and contextually constricted, often overlooking multiple concurrent and sequential interdependencies. Basic laboratory science functions largely according to linear chains in which functional relationships are established between certain variables. Single factors are usually identified as independent variables, which exert functional control over single dependent variables. However, the subject's behavior in a natural setting is guided by patterns of multiple concurrent and sequential stimuli. Drawing from Gottman and Roy's (1990) sequential analysis illustrations, a comparison of conditional and unconditional probabilities of multiple behavioral events as they flow through time may be a valuable analytic tactic. Knowledge of the temporal characteristics of prior events as they have naturally occurred may assist in the kind of prediction desired in science. It seems, therefore, that the nature of classroom instruction may be more fruitfully assessed as an interbehavioral field comprised of distinguishable but inseparable historical, setting, contextual, and behavioral elements that act to simultaneously and conjointly define the whole.

Redefining the Instructional Setting

The application of linear assumptions to the study of whole classrooms has been conceptually unsatisfactory and epistemologically forced. Multiplistic occurrences of a host of interrelated events are characteristic of instructional settings (Scarr, 1995). Altman (1988) and Altman and Rogoff (1987) argued that instructional elements mutually define one another. They understood the instructional whole as a host of inseparable aspects that serve to mutually explain one another in context. These aspects include the historical characteristics of teachers and students, the setting, the lesson content and activities, and the students' and teachers' current behavior. All of these contribute to the "meaning and nature of ... the [instructional] event' '(Altman& Rogoff, 1987,p. 24). It may readily be discerned, therefore, that an interbehavioral field systems perspective and the emergent view of instruction in a natural context are conceptually compatible.

The question, therefore, becomes "How, then, can such a rich setting be studied, and what questions ought its study be able to answer?" (Salomon, 1991, p. 14). Attempts to redefine a compatible research and evaluation process according to a field systems approach are scarce. Ray and Delprato (1989) offer one of the few generic behavioral systems methodologies available in the literature. A few recently available tactical guides are also more relevant to instructional settings (Frick, 1990; Greenwood et al., 1991; Greenwood, Cm% & Atwater, 199 1; Sharpe & Hawkins, 1992a, 1992b). We now turn our attention to a methodological process that draws from these seminal efforts.

An Assessment Protocol

A technology that enables concurrent evaluation of people, settings, and activities within particular instructional episodes shows great promise for educational improvement at a local level. Toward this end, we now describe one instructional measurement system (Sharpe, Bahls, & Wood, 199 1) that has been used with undergraduate teacher education students who teach in physical education and special education movement skill settings. [Currently, this evaluation instrument is used by the West Virginia University and University of Nebraska-Lincoln undergraduate teacher certification programs in physical education, and it is in the first cycle of implementation and possible adoption throughout the Lincoln Public School System. Experienced practitioners, university faculty, and select graduate students are working cooperatively on evaluation system training, preservice, teacher instruction, and on-site assessment (cf., Sharpe et al., 199 1). ]

Technological Foundations

The past 20 years may be characterized as a period of rapid technological evolution with readily apparent effects in scientific, technical, and industrial fields. Technology's effect on education in general and teacher education in particular, however, is less discernible (Hawkins & Wiegand, 1987). It is possible that the greatest potential for the application of technology to education may lie in the area of instructional measurement. The assessment of teacher and student process variables as they occur within the context of particular learning environments may be amenable to fresh technological approaches. The recurrent challenge is to find instruments that enable behavioral, ecological, and historical evaluation of particular classroom settings. We believe that this challenge, if met, may lead to direct improvement of educational programs at their most basic level--moment-to-moment teaching/learning activities.

Technological enhancement may make significant contributions to at least two areas in instructional measurement: more exhaustive recording and measurement capabilities in live instructional settings, and instructional evaluation based upon more sensitive data collection. Greater understanding of the variables extant in instructional settings, when coupled with an analysis of the functional relationships among those variables, may enable more effective pedagogical assessment.

Category Systems

When category systems are used to describe and analyze instruction, it must be remembered that such categories are somewhat arbitrary delineations of certain stimulus and response classes. In other words, the sum total of behavioral and contextual elements is subdivided into functionally defined groups that are thought to have a relationship to teaching effectiveness according to the informed opinion and literature of teacher educators. It still remains to be seen whether the categories in our example, or in any category system for that matter, are profitable in understanding the relationships in a learning environment. Categories often evolve into multiple subcategories, while others are synthesized to better describe and analyze the relationships in instructional settings. The number of categories that constitute a thorough description of an instructional system must also be balanced against a logistical requirement. In essence, the number of categories must be carefully weighed against the information gained and the ability of observers to collect it. System parsimony enhances the accuracy and reliability of the data collected.

The Measurement System

The Behavioral Evaluation Strategy and Taxonomy (BEST) and its companion, the Temporal Analysis System (TAS), were developed in concert by the faculty at the Teachers College at the University of Nebraska-Lincoln and the West Virginia University Sport Pedagogy faculty (Sharpe, Bahls, & Wood, 1991; Sharpe, Hawkins, & Wood, in press). Its purpose is to provide a rich empirical source of information from which instructional assessment may be made. BEST was designed to be used by teacher educators in evaluating preservice and inservice teachers in different subject matters. The system incorporates an extensive category system, including teacher and student behaviors, ecological elements, and historical factors. In addition, it can incorporate new or unique situational elements as the user observes them.

For illustrative purposes, we've chosen one category system used in physical education and special education movement skill settings. For the purpose of multiple evaluations across a similar practicum, a more simplistic code drawn from the BEST instrument was used. Table 18.2 displays this category system, which was implemented across several teacher certification practica and inservice settings. Comparison of Tables 18.1 and 18.2 should illustrate how instructional measurement tools for particular contexts may evolve from a more global category system.

Analysis of patterns in time. The analysis of patterns in time describes some of the relationships among contextual and behavioral events as they occur temporally. As Table 18.3 displays, frequency, conditional probability (the probability with which a particular subcategory either precedes or succeeds another), and statistical significance (based on conditional and unconditional probabilities, Gottman & Roy, 1990) are represented for preceding and succeeding events around a central subcategory of interest. Pattern-in-time searches may be programmed using preceding events, succeeding events, origination events, and length-of-chain parameters. Then, they may be searched individually or as a logically specified group. For example, all event chains of three that begin with a particular teacher behavior subcategory may be examined (e.g., all behavior chains that begin with verbal instruction and include other key instructional behaviors, such as verbal instruction --> specific observation --> positive feedback). All conceivable chains of two may be explored and represented by a matrix. All subcategories that either precede or succeed a particular subcategory may also be displayed (e.g., all behaviors that precede and succeed teacher modeling). Lagtime specifications may be specified in a pattern-in-time search (a time parameter between the onset of a central event and the onset of others); this is particularly useful if multiple ongoing events obscure a temporal chain of interest.

Goal setting. An integral component in our use of the BEST system is data-based goal setting, Approaches based on the contiguous temporal relationships among behavioral and contextual elements, and that make a direct connection with student responses, hold much promise for educational evaluation. An interbehavioral evaluation strategy seeks to capture such teacher <--> student relations within the larger umbrellas of contextual events and historical antecedents. Therefore, following assessment, goals are provided that relate directly to the discrete element summation and to the analysis of patterns in time. Of course, future observations may assess the degree to which goals are attained.

A caution. A major criticism of a behavior-analytic approach to assessment lies in its perception as a largely technocratic undertaking, driven more by advanced technologies employed by unreflective technicians than by thoughtful professionals (Schempp, 1987). However, it should not be assumed that particular quantitative levels or specific temporal relationships may be ascribed and extended to effective instruction. Rather, a technical tool merely enables a more representative and thorough picture of complex instructional episodes from a dynamic field systems perspective. A field systems procedure, and others that are akin, still requires the interpretive expertise of an experienced, trained educational evaluator. An interbehavioral system simply provides descriptive and analytic techniques for extracting behavioral and contextual information from instructional settings. The purpose is always to understand more fully the dynamics of complex educational environments.

To illustrate the value of a field systems approach to instructional measurement, we present two analyses of instructional performance change for selected events within Table 18.2.

Study I

Study I was a longitudinal analysis of different dimensions of selected teacher and student behaviors of physical education and special education majors in the University of Nebraska-Lincoln (UN-L) teacher certification program (N = 15). Aggregate data were used for each graph and instructional context and subject matter activities were matched cross- sectionally and longitudinally.

Preservice teachers in the undergraduate certification program were familiarized with the conceptual nature of an interbehavioral field systems approach to instructional measurement in the didactic portions of their introductory and intermediate methods classes. They were then repeatedly assessed according to the Table 18. 2 protocol throughout each practicum, student teaching, and in their initial inservice experiences. One aggregate early, mid, and end-of-experience data point for each practicum (i.e., methods practica, student teaching, and inservice) is represented for illustrative purposes in Figures 18.1, 18.2, and 18.3.

Figure 18.1. Percentage of total class time for selected teacher and student behaviors from the Table 18.2 assessment taxonomy.

As is readily discerned from Figures 18.1 through 18.3, aggregate trends were demonstrated across all variables in the recommended directions. Though only three aggregate points are represented within each practicum, each graph indicates the efficacy of the assessment protocol with regard to improvement both within and across phases for each variable targeted for change. Of additional interest is the field system orientation's ability to train, alter, and maintain more complex analytic units (e.g., sequences of teacher behaviors).

Figure l8.3. Conditional probability for trained teacher behavior analytic units from the Table 18.2 assessment taxonomy.

Study II

Study II involved an aggregate comparison of employed teachers who had been repeatedly exposed to the evaluation process in their undergraduate program (as in Study I, N = 15) and a similar inservice teacher 3 group that had not been similarly exposed (N = 15). [ Data collection and analysis was made possible by the Oliver E. Bird Fellowship for the UN-L Training and Generalization Project: A Behavioral Evaluation Strategy and Taxonomy for InService Teacher Evaluation, and UN-L Research Council Grant LWT/l 0-215-91601 Real- Time Data Analysis Software Applications.] Over the course of one academic year, a series of ten profiles using the BEST instrument were undertaken at regular intervals for each subject in representing the overall performance for the two groups.

Figure l8.4. Comparison of percentage of total class time devoted to selected teacher behaviors for the evaluation aggregate.

Figure l8.5. Comparison of percentage of total class time devoted to selected teacher behaviors for the control aggregate.

These findings demonstrate the usefulness of the BEST instrument as a measurement system for teacher education. More important, however, is the use of field systems analyses to understand and develop the temporal relationships among more complex clusters of teacher and student behaviors. Figure 18.6 shows percentages of class time devoted to Student success (i.e., accomplishment of a stated subject matter-related task--motor-appropriate behavior), whereas Figure 18.7 portrays the percentage of student engaged behavior (i.e., attempting a subject matter-related task, but doing so unsuccessfully--motor-inappropriate behavior) for the exposed group across sessions. The findings also represent the probabilities with which selected teacher behaviors evidence themselves in a manner temporally contiguous with the student variables.

Figure l8.6. Conditional probability of select teacher behaviors occurring in temporally contiguity as a function of the percentage of classtime in which motor-appropriate behavior is evidenced.

Figure l8.7. Conditional probability of select teacher behaviors occurring in temporally contiguity as a function of the percentage of classtime in which motor-inappropriate behavior is evidenced.

The evaluation process fostered interactions that enabled larger percentages of student success with diminished portions of student engagement over the course of the year. Further, teacher proximate (i.e., the teacher in close proximity to a particular student) and individual teacher focus (i.e., the teacher attends to a particular student as opposed to student groups) appear to function as temporal facilitators of student success, as indicated by the heightened conditional probability levels that correlate positively over time with levels of student success. As similar pattern was also seen between teacher distant and general focus probabilities and student engaged (unsuccessful attempts), though the levels of student engaged declined across sessions.

When combining student success with student-engaged class percentages across graphs (i.e., the total time students spend in subject matter attempts regardless of success), the evaluation procedure may also be focused on classroom practices, which enable greater portions of time to be devoted to subject matter practice. This was particularly salient in light of the diminished portions of total class time devoted to managerial concerns across the academic year for the experimental aggregate (refer to Figure 18.4).

From these data, it seems that behavior analysis is increasingly capable of examining some of the more complex response classes of human subjects in educational settings using an interbehavioral technology. This has largely been due to a field systems perspective that has emerged contemporaneously with growing computer technologies. The combination has made possible the technological application of a theoretical framework, which has enabled a more complex data collection, analysis, and evaluation process.

Implications and Applications

It is increasingly clear from this study and the others referenced that technology is enabling behavior analysts to understand, and even control, more complex response <--> stimulus configurations. This provides an impetus for evaluating and altering complex behavioral and contextual interactions in classroom settings. If this is the case, then two questions arise: Should we do what is technologically feasible? and Which configurations should be taught now that we know we are capable? Though it is easy to become mired in the particulars of which educational ends justify which technological means, the larger question of the general effects of technology must be considered first.

Therefore, what purpose may we legitimately expect to have served by enhancing our technological understanding of excellence in teaching? Is it to remain merely a tool for assessment, or may we anticipate applying it to the enhancement of other dimensions of the teaching process, including perhaps the teaching of the prospective teacher (i.e., applying technology to teacher education as well as to evaluation)? Several issues seem to be raised by such a vision, not the least of which is the possibility of computer-driven simulations of the process. Perhaps the most realistic form that such simulation might take is that of an interactive video.

If our interest is in developing such a technology, data must come from a variety of experts doing a variety of things before we attempt to articulate any single approach or strategy appropriate to any given circumstance. In addition, variations between individual students help define what is likely to work and what isn't. Expert teachers not only fail when they try to teach all things, they also fail sometimes in teaching their strongest areas to some select students. But what is the feasibility for bringing simulation technology into such a complex process?

Given the fact that data-based assessment programs such as those reported earlier will likely generate a variety of concrete examples of good and not-so-good teaching over a variety of student learning and setting variations, what if the critical features of these various episodes were reconstructed on videodisc as isolated "scenes" for eventual computer-selected simulation application? Such a corpus of video scenes might well illustrate most, if not all, of the positive and negative event features that new teachers are likely to experience, including the individuals and ecological settings with which they will eventually deal.

First, real databases could be used to "seed" the simulation's onset, thereby allowing the simulator to determine sequential transitions from scene to scene. This would be accomplished by following in the same stochastic sequence determined by the type of data being simulated (e.g., poor teaching interaction databases, better teaching databases, and expert teaching databases).

Yet another variation on this computer-operated video simulator could allow for the viewer to interact with the sequence playback by using keyboard inputs. In this scenario, viewers could watch poor teaching sequences until they felt they recognized the types of problems being demonstrated. Then they could intervene to shift the driving stochastic programs to more-effective strategies by interceding with better-interaction selections. As a result, they could "teach" the teacher being observed on the video to use different teaching techniques, thereby learning what really is, and is not, a "better" technique by observing student performance outcomes reported via the simulator's play sequence.

The critical factor in all good simulators is whether an expert observer (such as those doing the coding in the evaluation studies) can detect whether a true human expert is in control, or whether a novice or learner is in control. As such, all interactions with the interactive-video simulator should be automatically "coded" by the simulator and used to contrast with the expert teacher's choices of interaction with the training simulator. When they are statistically indistinguishable, the student has personally learned to "simulate" the expert. When such a performance level has been reached, the student should be placed into real-life situations to train generalization.

The challenge lies not only in recognizing the task that lies before us in trying to describe and evaluate good teaching, but also in recognizing the tools that are already available for accomplishing that task. Scientific description is of little use if we cannot use it to reconstruct the original sequences and dynamics

of events being described. Computer and interactive video simulation is, after all, merely a reconstructive process that makes the original events appear to be real-time depictions to those observing and describing the simulation. Eventually, simulation becomes our own quality check on the reality of scientific progress: if we understand and describe the attendant variables well enough to make our simulations appear real, we probably have the technology necessary to use it to train new observers and participants in the process.

In line with Greenwood, Delquadri, Stanley, Terry, and Hall (1985), we recommend four areas for application of these emergent assessment strategies: (1) the development of a comparative database derived from different instructional settings as temporal conglomerates of contextual and behavioral variables; (2) the implementation of subsequent causal analyses of relationships that frequently appear in such databases; (3) the monitoring of the fidelity of interventions in specific contexts based on causal conclusions; and (4) the assessment of long-term changes in contextual and behavioral dependencies that have resulted from these interventions.

Greenwood, Carta, Arreaga-Mayer, and Rager (1991) summarized the implications of instructional evaluation efforts comparable to the one described:

The utility of a search and validate approach to the evaluation of effective instructional practices has only just begun to be undertaken. Because of its ... analysis of classroom behavior, and temporally related situational features of classroom instruction, it is an approach consistent with current school improvement goals. It is also an approach that focuses on the contributions classroom teachers can bring to the development of effective instructional technology. Clearly, demonstrations of the approach are warranted. [pp. 188-1891

As the technological revolution approaches educational evaluation, careful consideration of the empirical and ethical limits of technology must be undertaken through the scholarly dissemination of its functional possibilities. The end of more thorough and accurate educational evaluation strategies will necessarily combine calculated experimentation with emergent technology.

References

ALTMAN, 1. (1988). Process, transactional, contextual, and outcome research: An alternative to the traditional distinction between basic and applied research. Social Behavior, 3, 259280.

ALTMAN, I., & Rogoff, B. (1987). World views in psychology: Trait, interactional, organismic, and transactional perspectives. In D. Stokolis & 1. Altman (Eds.), Handbook of environmental psychology (pp. 1-40). New York: Wiley.

BRONFENBRENNER, U. (1979). Contexts of child rearing: Problems and prospects. American Psychologist, 34, 84-89.

DARST, P. W., Zakrajsek, D. B., & Mancini, V. H. (1989). Analyzing physical education and sport instruction. 2nd ed. Champaign, IL: Human Kinetics.

DAWE,H.A. (1984). Teaching: Social science or performing art? Harvard Educational Review, 54,111-114.

DICKIE, R. F. (Ed.). (1989). The Juniper Gardens Children's Project. Education and Treatment of Children [Special monograph issue], 12(4).

DILLON, W. R., Madden, T. J., & Kumar, A. (1983). Analyzing sequential categorical data on dyadic interaction: A latent structure approach. Psychological Bulletin, 94, 564-583.

EISNER, E. W. (1983). The art and craft of teaching. Educational Leadership, 40, 4-13.

ELLUL, J. [19541 (1964). The technological society. Trans. J. Wilkinson. New York: Knopf.

FINN, C. E. (1988). What ails education research. Educational Researcher, 17, 5-8.

FIRESTONE, W. A. (1997). Meaning in method: The rhetoric of quantitative and qualitative research. Educational Researcher, 16,16-21.

FRICK, T. W. (1990). Analysis of patterns in time: A method of recording and quantifying temporal relations in education. American Educational Research Journal, 27(l),180-204.

GAGE, N. L. (1978). The scientific basis of the art of teaching. New York: Teachers College Press.

GAGE, N. L. (1984). What do we know about teaching effectiveness? Phi Delta Kappan, 66, 87-90.

GARRISON, J. W. (1986). Some principles of postpositivistic philosophy of science. Educational Researcher, 15, 12-18.

GOTTMAN, J. M., & Roy, A. K. (1990). Sequential analysis: A guide for behavioral researchers. New York: Cambridge University Press.

GREENWOOD, C. R., Carta, J. J., & Atwater, J. (1991). Ecobehavioral analysis in the classroom: Review and implications. Journal of Behavioral Education, 1, 59-77.

GREENWOOD, C. R., Carta, J. J., Arreaga-Mayer, C., & Rager, A. (199 1). The behavior analyst consulting model: Identifying and validating naturally effective instructional methods. Journal of Behavioral Education, 1, 165-191.

GREENWOOD, C. R., Delquadri, J. C., Stanley, S. 0., Terry, B., & Hall, R. V. (1985). Assessment of eco-behavioral interaction in school settings. Behavioral Assessment, 7, 331-347.

HAWKINS, A. H., & Wiegand, R. L. (1987). Where technology and accountability converge: The confessions of an educational technologist. In G. T. Barrette, R. S. Feingold, C. R. Rees, & M. Pieron (Eds.), Myths, models, and methods in sport pedagogy (pp. 67-75). Champaign, IL: Human Kinetics.

HOWE, K. R. (1988). Against the quantitative-qualitative incompatibility thesis or dogmas die hard. Educational Researcher, 17,10-16.

HUBERMAN, M. (1987). How well does educational research really travel? Educational Researcher, 16, 5-13.

JACKSON, P. W. (1968). Life in classrooms. New York: Holt, Rinehart & Winston.

KANTOR, J. R. (1941). Current trends in psychological theory. Psychological Bulletin, 38, 29-65.

KANTOR, J. R. (1946). The aim and progress of psychology. American Scientist, 34, 251-263.

KANTOR, J. R. (1953). The logic of modern science. Chicago: Principia Press.

KANTOR, J. R. (1959). Interbehavioral psychology. Granville, OH: Principia Press.

KANTOR, J. R. (1969). The scientific evolution of psychology. Vol. 2. Chicago: Principia Press.

KANTOR,J.R.,& Smith,N.W. (1975). The science of psychology: An interbehavioral survey. Chicago: Principia Press.

KAZDIN, A. E. (1982). Single case research designs. New York: Oxford University Press.

MORRIS, E. K., & Midgley, B. D. (1990). Some historical and conceptual foundations of ecobehavioral analysis. In S. R. Schroeder (Ed.), Ecobehavioral analysis and developmental disabilities: The twenty-first century (pp. 1-32). New York: Springer-Verlag.

RAY, R. D. (1992). Interbehavioral methodology: Lessons from simulation. Journal of Teaching in Physical Education [special monograph issue].

RAY, R. D., & Delprato, D. J. (1989). Behavioral systems analysis: Methodological strategies and tactics. Behavioral Science, 34, 81-127.

RUBIN, L. J. (1985). Artistry in teaching. New York: Random House.

SALOMON, G. (1991). Transcending the qualitative-quantitative debate: The analytic and systemic approaches to educational research. Educational Researcher, 20, 10-18.

SCARR, S. (1985). Constructing psychology: Making facts and fables for our times. American Psychologist, 40,499-512.

SCHEMPP, P. K. (1987). Research on teaching in physical education: Beyond the limits of natural science. Journal of Teaching in Physical Education, 6(2), 111-121.

SHARPE, T. L. (1990). Field systems data: An exploration of alternative visual representations. Interbehaviorist, 18(2), 4-8.

SHARPE, T. L., & Hawkins, A. (1992a). Field systems analysis: A tactical guide for exploring temporal relationships in classroom settings. Teaching and Teacher Education: An International Journal of Research and Studies, 8, 171-186.

SHARPE, T. L., & Hawkins, A. (Eds.) (1992b). Field systems analysis: An alternative strategy for the study of teaching expertise. Journal of Teaching in Physical Education, 12, 1.

SHARPE, T. L., & Hawkins, A. (1992c). Field systems analysis: Prioritizing patterns in time and context among observable variables. Quest, 44, 15-34.

SHARPE, T. L., Bahls, V., & Wood, D. (1991). University of Nebraska teacher and student behavioral evaluation strategy and taxonomy (BEST). 3rd ed. Lincoln: University of Nebraska-Lincoln.

SHARPE, T. L., Hawkins, A., & Wood, D. (in press). Temporal analysis system software guide and users' manual. Ontario, Canada: S&K Computer Products, Ltd.

SHARPE, T. L., Bahls, V., Seagren, S. & Wolfe, P. (1991) A collaborative approach to practica based teacher preparation: The Nebraska - Lincoln Model. In G. M. Graham & M. A. Jones (Eds.), Collaboration between researchers and practitioners in physical education: An international dialogue (pp. 99-96). Champaign, IL: Human Kinetics.

SHAVELSON, R. J., & Berliner, D. C. (1988). Erosion of the education research infrastructure: A reply to Finn. Educational Researcher, 17, 9-14.

WITT, J. C. (1990). Complaining, preCopernican thought and the univariate linear mind: Questions for school-based behavioral consultation research. School Psychology Review, 19, 367377.