Home:  Reports Page:   LEE II - Varnhagen:

Final Report

Technologically Enhanced Supplements

for Introductory Psychology

Learning Enhancement Envelope (LEE) Initiative 11
October, 1998

Click Here to go to Table of Contents
Click Here to go back to the AToP Reports page

Compiled by

Connie K. Varnhagen
Department of Psychology
University of Alberta

Table of Contents

1. Name of Project
2. Brief Overview of Project
3. Deliverables from the Project
4. Project Planning and Implementation
5. Project Accomplishments
6. Evaluation Results
7. Lessons Learned and Recommendations
8. Means of Disseminating Results
9. Faculty Development
10. Future Plans and Spin-offs
11. Financial Reporting
Appendix A: Sample Web Pages and Syllabi – Psycho 104/105
Appendix B: intro.psych Brochure
Appendix C: Sample Web Pages – Reading the Research
Appendix D: Sample Web Pages: Literature and Psychology
Appendix E: Objectives and Goals of the Undergraduate Curriculum in Psychology
Appendix F: Final Report of the Second Annual Teaching of Psychology Conference
Appendix G: Financial Statements

Back to Top of Page
 
 

LEE I Final Report

October 15, 1998

1. Name of Project:Technologically Enhanced Supplements

for Introductory Psychology (Initiative #11)

Names of Project Members:
 
Faculty:  Graduate Students:
Connie Varnhagen
Don Heth
Don Kuiken
Gay Bisanz
Department of Psychology

David Miall
Department of English 

Leanne Willson
Patricia Boechler
Jackie Leighton
Michael Snyder
Juliette Klein
Department of Psychology

Address on Campus:
Connie Varnhagen
Department of Psychology
P220 Biological Sciences Building

Back to Top of Page
Back to Table of Contents

2. Brief Overview of Project

Nearly 10,000 students enrolled in Alberta post secondary institutions complete introductory psychology courses each year. These courses are all very different – they are of different lengths (e.g., one or two terms), use different texts, and are taught in different ways (ranging from individual study to 500-student lectures). Nevertheless, these courses cover an almost identical set of topics. The objectives of this multi-phase, multi-year, collaborative project are to develop and evaluate technologically enhanced supplements to enhance teaching and learning of common topics in introductory psychology.

The first phase, supported by LEE I funding, was to develop a number of World Wide Web-based modules for developing critical thinking, reading, and writing skills while providing greater exposure to common topics in introductory psychology. The second phase, supported by LEE II and LEE III funding, includes developing collaborative links between Alberta post secondary institutions and with major North American introductory psychology textbook publishers to (a) expand development and evaluation of technologically enhanced supplements, (b) integrate these supplements into alternative, technologically enhanced introductory psychology courses, and (c) evaluate customized arrays of these supplements in contrasting environments.

This report summarizes development and initial formative evaluation of Web-based modules accomplished during the March, 1997 – September, 1998 LEE I funding period.

Back to Top of Page
Back to Table of Contents

3. Deliverables from the Project

The main deliverable from this project consists of intro.psych, the two experimental, technologically enhanced courses. PSYCO 104 (Basic Psychological Processes), spearheaded by Don Heth, is being piloted with a limited number of students Fall term, 1998-99, and will be offered to a larger number of students Winter term. Components of PSYCO 105 (Individual and Social Behaviour), spearheaded by Connie Varnhagen, was piloted by Leanne Willson in Spring term, 1997-98; PSYCO 105 is being piloted with a limited number of students Fall term, 1998-99, and will be offered to a larger number of students Winter term. The home pages and syllabi for these two courses is attached as Appendix A. The URL for PSYCO 104 is http://web.psych.ualberta.ca/~dheth/Intro_104/ htdocs/Home.htm; the URL for PSYCO 105 is http://web.psych.ualberta.ca/~varn/
EZSite/Home.htm.

Connie Varnhagen developed a brochure advertising intro.psych, with assistance from Patricia Boechler and the Production Studio of Academic Technologies for Learning. This brochure was distributed through the bookstore in the summer of 1998 and will be distributed to high school guidance counselors in fall of 1998. A copy of this brochure is attached as Appendix B.

Reading the Research is a module for helping students learn how to read published empirical research articles. Connie Varnhagen originally developed it as part of intro.psych but the module can be adapted easily for use in many other psychology courses. Reading the Research was evaluated in PSYCO 208 (Experimental Methods) at Grant MacEwan Community College by Connie Varnhagen and Nancy Digdon (GMCC), Winter term 1997-98. In addition to its use in the intro.psych courses, Reading the Research is being used in a number of courses at Grant MacEwan Community College, Keyano College, and Grand Prairie Regional College, beginning Fall term 1998-99, and is being licensed to Harcourt Brace Publishers for use at their Web site. Sample pages for an article summarized in Reading the Research is attached as Appendix C. The URL for this sample is http://web.psych.ualberta.ca/~varn/EZReading-SocDev/Reading.htm.

Psychology and Literature is an interdisciplinary offshoot of intro.psych. As described in 3. Project Planning and Implementation, we are beginning to reform and revitalize the undergraduate curriculum in the Department of Psychology. One goal of our process is to help students realize interdisciplinary links between psychology and other disciplines. Psychology and Literature is being developed by David Miall (English) and Don Kuiken to (a) foster student understanding of interpretive studies in psychology and in literature; (b) provide opportunities for interactive identification - and refinement - of the interpretive approach that the student initially brings to the text that is being interpreted; and (c) encourage inter-disciplinary consideration of the common features of interpretive studies in literature and psychology. Sample pages for an interpretive study of the poem, "Frost at Midnight," by S. T. Coleridge, is attached as Appendix D. The URL for this sample is http://www.ualberta.ca/~dmiall/psychlit/psychlit.htm. Work is underway for a parallel interpretive module concerning Freud’s well-known specimen dream, "The Irma Dream."

Back to Top of Page
Back to Table of Contents

4. Project Planning and Implementation

Faculty in the Department of Psychology began a process of curriculum reform during the 1996-97 academic year. To guide curriculum reform, the Department of Psychology Faculty Council passed a statement of Objectives and Goals of the Undergraduate Curriculum. This statement is included in Appendix E and is published on our departmental Web site, http://web.psych.ualberta.ca/mission_statement.htmld/index.html.

Don Kuiken, Connie Varnhagen, Don Heth, and Gay Bisanz developed a draft set of objectives for knowledge, skills, and attitudes undergraduates should acquire from our introductory psychology courses. These objectives are being revised, will be distributed to introductory psychology instructors for comments and revisions, and will be brought to Faculty Council for consideration for adoption during the 1998-99 academic year. These objectives for introductory psychology are meant to serve as a model for curriculum reform in the department leading to a more precise specification of knowledge, skills, and attitudes that undergraduate students should acquire as a part of the psychology major.

Connie Varnhagen, Don Heth, and Don Kuiken are using the objectives for the undergraduate curriculum and the draft objectives for introductory psychology to guide development of technologically enhanced supplements for teaching introductory courses and concepts in psychology.

We have adopted a team approach that recognizes the unique contributions to be made by each member: Connie Varnhagen has developed the general approach to offering technologically-enhanced alternative courses on the Web and has developed specific content for these courses; Don Heth has addressed the technological aspects of mounting such courses, including user authentication and site security; and Don Kuiken has developed interdisciplinary links essential for providing an integrated liberal arts education.

We have also involved graduate students in all aspects of the project: Leanne Willson and Juliette Klein assisted Connie Varnhagen in developing course content, Leanne Willson piloted the content during Intersession, Jackie Leighton and Patricia Boechler are assisting Connie Varnhagen and Don Heth in offering and evaluating the experimental courses, and Michael Snyder assisted Don Kuiken in developing content for the interdisciplinary module.

Finally, we needed to upgrade the existing departmental computer network to support offering technologically enhanced courses. Connie Varnhagen and Don Heth met often with Gary Finley, Faculty Service Officer, Department of Psychology, to consult on server upgrades, computer access, programming, and training needs in the department.

Timeline for activities completed by the principle project members during the first phase of our project:
 
Component Fall, 1997 Winter, 1998 Spring/Summer, 1998 Fall, 1998
Content

(Varnhagen)

  • Web sites and critical thinking exercises for PSYCO 105
  • Reading the Research
  • Web sites and critical thinking exercises for PSYCO 104
  • Evaluate Reading the Research
  • Evaluate Sites to Visit (PSYCO 105)
  • Develop PSYCO 105
  • Write LEE III proposal
  • Coordinate ATOP meeting
  • Pilot PSYCO 105
  • Evaluate PSYCO 105
  • Modify PSYCO 105
  • Programming

    (Heth)

    • Web access tracking
    • Efficiencies and drawbacks of different languages
  • Logging scripts
  • Redirection scripts
  • Encryption
  • Develop PSYCO 104
  • Pilot PSYCO 104
  • Evaluate PSYCO 104
  • Modify PSYCO 104
  • Inter-disciplinary

    (Kuiken)

    • Develop literature module
    [on study leave]
    [on study leave]
    • Develop psychology module

    In addition to the specific activities described above, all three faculty members meet weekly with Gay Bisanz (Acting Chair and another project member) to develop objectives for introductory psychology and engaged in faculty development and consultation activities. As well, Connie Varnhagen, Gay Bisanz, Don Heth, Don Kuiken, and other members of the department participated in a technology development and integration workshop developed by Michael Szabo, Faculty of Education, as part of the Training, Infrastructure, and Empowerment System for Professional Development (LEE Initiative #17) in February, 1998.

    Connie Varnhagen helped organize and presented at the second meeting of the Alberta Teachers of Psychology (the final report is attached as Appendix F); Leanne Willson presented her work at the meeting and Don Heth, Patricia Boechler, Jackie Leighton, and Michael Snyder attended the conference. Finally, Connie Varnhagen engaged in developing collaborative links with other Alberta post secondary institutions, grant and report writing, conference presentations, and promotion of intro.psych.

    Back to Top of Page
    Back to Table of Contents

    5. Project Accomplishments

    The deliverables described above (see 2. Deliverables) and planned for the second and later phases of this project are based on (a) developing our instructional framework based on psychological theory and empirical findings, (b) developing the technology to support our instructional framework, (c) identifying, modifying, and developing content for the intro.psych courses, and (d) collaborating on research and development with departments and faculties at the University of Alberta, other post secondary institutions in the province, and textbook publishers.

    During the 1998-99 academic year, Don Heth and Connie Varnhagen are evaluating the experimental intro.psych courses, examining the framework, technology, and content through attitude surveys, behaviour, and learning outcomes. At the same time, we are leading further development in content and technology, consulting with colleagues on curriculum change and technology, and developing and extending collaborative links within and outside of the university.

    Meeting the Objectives of the Learning Enhancement Envelope

    • Expand Opportunity. Development and evaluation of alternative, technologically enhanced courses in introductory psychology supports both objectives of expanding opportunity, namely providing alternative forms of access to learning materials and allowing for access to information from a distance. We anticipate that instructors working with different texts and in diverse educational settings across the province will be able to customize learning supplements to fit their students’ learning styles, their own particular teaching objectives and style, and their institution's available infrastructure. In addition, our emphasis on asynchronous communication and learning modules delivered via the Internet supports residential as well as distance education.
    • Improve learning. This objective of improving student learning is the driving force behind our long-term collaborative project. Large lectures and dwindling resources severely restrict the ability to provide introductory students with rich learning experiences. A large body of previous research indicates that virtual learning experiences and enhanced communication facilitate student learning. As well, our framework of the learner and emphasis on research and evaluation allows a progressive differentiation between aspects of learning that are best fostered through technology-based applications and those that distinctly require direct person-to-person interaction.
    • Increase productivity. The collaborative nature of the second and later phases of this project will allow instructors to realize economies of scale that could not be possible in individual projects. Communicating the results of our development and evaluation to commercial developers will inform the development and implementation of sound technology-based approaches to learning. We are already beginning to see this in our interactions with other institutions and publishing companies during this phase of the project. Furthermore, we have now established firm collaborative links with five other Alberta post secondary institutions (as well as informal links with most other post secondary institutions offering introductory psychology courses) and major North American publishing companies as part of our successful LEE III collaborative grant.
    Back to Top of Page
    Back to Table of Contents

    6. Evaluation Results

    Needs assessment and formative evaluation have been conducted as part of the first phase; presentations based on this research are listed below (see 7. Disseminating Results).

    As a part of our needs assessment, we surveyed 910 introductory psychology students regarding their access to computers and self-reported computer skills. Although 88% of students had access to a home computer and 65% had access to the Internet, only 20% rated themselves as very comfortable using computers, 26% as having good to excellent skills at using E-mail and 20% as having good to excellent skills at using the World Wide Web. Thus, although students have at least some access to computers, they do not report very good computer skills. The results from this survey prompted us to offer the experimental intro.psych courses only to students who considered themselves to be skilled at using the Internet.

    Students represented a range of skill in the classes in which we conducted formative evaluation using various components of intro.psych. Varnhagen and Digdon (1998) examined the use of Reading the Research in Psyc 208, Experimental Psychology, at Grant MacEwan Community College. Students expressed generally positive attitudes toward and indicated learning from the modules. Interestingly, although students who were more comfortable with computers expressed a more positive impression of the modules they used, students who were less comfortable with computers reported enjoying the module more.

    Varnhagen and Willson (1998) found similar results in a their evaluation of Sites to Visit. In addition, consistent with other research, although we developed Help features, students seldom used them. Students rated access to lecture notes as very useful and almost all students printed out the lecture notes prior to attending lectures. Students also preferred the format of greater opportunity to explore topics in introductory psychology on their own time coupled with fewer lectures.

    Our evaluation of the experimental courses this year will address four primary questions in the following ways:

    (a) What are students’ attitudes toward intro.psych? We are administering attitude questionnaires at the beginning, throughout, and at the end of the term. Items were adapted from a database of items maintained by Stanley Varnhagen, Evaluation Researcher, Academic Technologies for Learning. Attitudes toward computers and technologically enhanced courses, in general, and different components of the course (e.g., the text, the lecture, Sites to Visit, Reading the Research, Help), in particular, are being assessed. It is important to measure attitude as part of the evaluation because of the strong relationships between student attitude, motivation, and performance.

    (b) How do students use intro.psych? Our departmental server software and the record keeping system that Don Heth developed allow us to track students’ use of intro.psych. We will be examining student sessions to determine what types of sessions students engage in, when they engage in these sessions in relation to the course schedule, how they navigate, and how they interact with other students in their discussion groups, the instructor, and the graduate teaching assistant.

    This research is essential not only to examine our assumptions about student use upon which we based our development of intro.psych, but also to better understand how people navigate through and learn from the Web. As experimental psychologists with specific expertise in learning theory, information processing, and instructional psychology, Don Heth, Connie Varnhagen, Jackie Leighton, and Patricia Boechler comprise a strong research team for tackling issues in learning from the Internet as well as evaluating our LEE-sponsored intro.psych courses.

    (c) What do students learn from intro.psych? Although there is a great deal of overlap, some course material is specific to a particular learning component of the course. For example, in PSYCO 105, some tested information is specific either to the text, the lectures, or Sites to Visit. Performance on fact/comprehension questions and application/

    analysis/evaluation questions will be compared as a function of course component. We anticipate finding an interaction between type of question (fact/comprehension versus application/analysis/evaluation) and learning component.

    We are also developing a critical thinking test to administer to students enrolled in various introductory psychology classes. One important goal motivating development of intro.psych is to enhance critical thinking skills. We will compare critical thinking by students enrolled in traditional lecture courses with students enrolled in intro.psych to determine whether intro.psych enhances critical thinking skills.

    (d) How do students’ attitudes and use relate to learning from intro.psych? Based on the rich research literature on student learning, we expect to find a number of relationships between our measures of student attitude, use of intro.psych, and student learning.

    A hypothetical model of the relationships between these variables is:

    Student variables that we are measuring include gender, faculty, year, motivation, attitude, and discussion group to which they have been assigned. Technology variables include computer access and skill. Instructional variables include format of content presentation (text, lecture, Web), topics, time/frequency of sessions on the computer, gets/puts, and number of words posted across weeks. Outcome variables include knowledge, skills, and attitudes about introductory psychology, marks, performance on the critical thinking test, motivation to take other psychology courses, and progress through their degree program.

    We will examine the relationships among these measures not only as part of our evaluation of intro.psych but to advance understanding of instruction and learning.

    Back to Top of Page
    Back to Table of Contents

    7. Lessons Learned and Recommendations

    Although all project members entered into the process with a great deal of experience in teaching, research, and project management, we all were rather naïve about the enormous expenditure of time and effort is necessary to consider curriculum reform, develop new curricula, and implement the new approach. Anyone deciding to undertake such a large-scale project must be ready to commit time and effort, and not attempt to simply "fit" the extra work into the already overburdened schedule of university research, teaching, and service.

    Another lesson learned was that it is important to have a departmental center for technology development and evaluation. The Production Studio is the ideal environment for major development, but smaller-scale resources must be available in the home department. To address this issue, we have successful obtained financial support from the Faculty of Science to jointly sponsor an Industrial Internship Program student to assist with development and evaluation. We have also recently purchased a departmental computer and software to assist faculty in development.

    Finally, we have discovered that many other faculty members, departments, and faculties have begun to engage in technology development without a clear vision or set of objectives to guide development. We have begun to argue the need for clear objectives to guide curriculum reform, not merely some push to engage in technology development. We used the TIES workshop and meetings prior to and following the workshop to formulate our long-term vision for curriculum reform in the Department of Psychology undergraduate program. We anticipate that our project will ultimately meet the LEE objectives and have an impact on teaching and learning because it is driven by our objectives for the undergraduate curriculum.

    One recommendation that has been repeated stressed by University administrators concerns the role of technological innovation in the normal career development of university professors. University professors in the Faculties of Arts and Science are scholars, teachers, and researchers. The amount of work required to develop not just new courses, but new technologies for the courses, must be justified within this framework if these initiatives are to be sustained. Psychologists are fortunate in that the impact of these technological changes is of interest within the discipline itself. We have therefore developed within our approach a central focus on viable research questions and issues that could inform the wider psychological community. This is shown in our model of evaluation described in 5. Evaluation Results. We think that much of our experiences will ultimately be publishable in peer-reviewed journals that address issues of pedagogy and technology. These opportunities for professional development will, we believe, expand the collaborative nature of the project to the wider community of post secondary education.

    Back to Top of Page
    Back to Table of Contents

    8. Means of Disseminating Results

    We anticipate presenting our process and research at informal seminars and presentations across campus, at local and provincial meetings, and at peer-reviewed national and international conferences. We anticipate publishing our work in peer-reviewed journals, such as Teaching of Psychology, a journal sponsored by the American Psychological Association.

    Varnhagen, C.K. (1997, November). Evaluation of Reading the Research. Presentation to ShowTime, Academic Technologies for Learning, University of Alberta. [slides available at http://web.psych.ualberta.ca/~varn/ShowTime/Goals.htm]

    Varnhagen, C.K. (1998, June). Maybe we should resist: Grappling with issues in technology and instructional innovation. Presentation at the meeting of the Association for Media and Technology in Education in Canada, Edmonton, AB. [slides available at http://web.psych.ualberta.ca/~varn/AMTECs/sld001.htm]

    Varnhagen, C.K., and Digdon, N. (1998, June). Reading the Research: Helping students learn how to read empirical research reports. Presentation to the Second Annual Alberta Teachers of Psychology meeting, Edmonton, AB. [slides available at http://web.psych.ualberta.ca/~varn/Nancy-s/sld001.htm]

    Varnhagen, C.K., and Willson, L. (1998, June). Using the Internet to enhance critical thinking in introductory psychology. Presentation to the Second Annual Alberta Teachers of Psychology meeting, Edmonton, AB. [slides available at http://web.psych.ualberta.ca/~varn/Leanne-s/sld001.htm]

    Back to Top of Page
    Back to Table of Contents

    9. Faculty Development

    Faculty and graduate students attending workshops offered through Academic Technologies for Learning, attended Show Time presentations at the Production Studio, and consulted with staff in the Production Studio on a regular basis. The development to date, as well as the planned additional work, would not have been possible without the support of the Production Studio and Academic Technologies for Learning. Faculty also participated in a weeklong TIES workshop (described above in 3. Project Planning and Implementation), considering long-range planning for and implementation of curriculum reform. Faculty, graduate students, and other members of the department attended and participated in the second annual meeting of the Alberta Teachers of Psychology (described in 3. Project Planning and Implementation and attached as Appendix F).

    Back to Top of Page
    Back to Table of Contents

    10. Future Plans and Spin-offs

    We were successful in obtaining LEE II funding with the University of Calgary to begin work on the second phase of the project, namely to develop and evaluate technologically enhanced supplements for introductory psychology. This phase has been expanded to six formally-involved post secondary institutions (University of Alberta, University of Calgary, Grant MacEwan Community College, Concordia University College of Alberta, Keyano College, Red Deer College) and two publishing companies (Harcourt Brace and Brooks/Cole-ITP Nelson) who are collaborating through LEE III funding and a number of more informally-involved institutions (including Augustana College, Canadian University College, and Grand Prairie Regional College). The ultimate goal of our multi-year, multi-phase collaborative project is to develop and support a technological resources library from which introductory psychology instructors from across Alberta may draw resources for enhancing teaching and learning in their psychology courses.

    As well, because of the experiences Connie Varnhagen acquired through LEE I funding opportunities, she has served as the Editor of Research and Teaching Focus, a Brooks/Cole Web site feature that showcases recent research for psychology students. Her development of this Web site (an example is found at http://psychstudy.brookscole.com/sample/

    student/common/symposium/index.shtml) is being used as a model for other Web-based supplements being developed for Brooks/Cole).

    In addition, Connie Varnhagen and Leanne Willson were invited to submit a proposal for developing a multimedia supplement to accompany a new edition of Gray’s Psychology, a major introductory psychology text published by Worth. Their proposal was selected from among a number of proposals invited from across North America and Connie Varnhagen has developed the first edition of Gray PLUS, a CD-ROM/Web site supplement for students and instructors. The Web companion portion of the development is found at http://www.worthpublishers.com/gray/. Proceeds from this work are being used to establish graduate scholarships in the Department of Psychology.

    Finally, the Faculties of Science and Arts have recognized and rewarded our work in the form of financial support. The Faculty of Science is jointly sponsoring (with our LEE III funding) an Industrial Internship student to provide support to faculty in developing and evaluating Web based supplements for psychology courses. The Faculty of Arts has provided funding for graduate teaching assistants for the experimental intro.psych courses offered during the 1998-99 academic year.

    Back to Top of Page
    Back to Table of Contents

    11. Financial Reporting

    Financial statements from March, 1998 and from September, 1998 are attached as Appendix G.
     
     
    LEE Funding
    University Funding
    Item
    Budgeted1
    Actual
    Budgeted7
    Actual
    Faculty Release Time
    70,000.00
    26, 930.002
      2,580.008
    Graduate Student Support
    20,000.00
    12, 882.253
      2,685.709
    Equipment  
    7,017.454
      10
    Software  
    952.514
      10
    Travel
    10,00.00
    475.485
       
    Miscellaneous  
    471.316
       
             
    TOTAL
    100,000.00
    48,729.00
      5,265.7011
             
    Committed 

    Duplicating

    Computer software

    Training

     
    1,271.00
       
       
    50,000.00
       

    1 The proposed LEE budget was based on two years of support. We did not revise the budget when we received support for one year.

    2 Breakdown:

    Connie Varnhagen $15,494.00 for 3 courses sessional replacement

    Don Heth 5,064.00 for 1 course sessional replacement

    Don Kuiken 4,122.00 for 1 course sessional relief

    David Miall (English) 2,250.00 for .5 course sessional relief

    In relation to the specified template items, Connie Varnhagen’s work was related to the following items:

    Content Expert, Authoring; Consultant, advisor (to other faculty); Project Management; Instructional Design; Internet Specialist; Graphics/Interface Design; Staff Development (to other faculty); Pilot; Evaluation

    In relation to the specified template items, Don Heth’s work was related to the following items:

    Content Expert, Authoring; Consultant, advisor (to other faculty); Internet Specialist; Graphics/Interface Design

    In relation to the specified template items, Don Kuiken’s work was related to the following items:

    Content Expert, Authoring; Project Management; Instructional Design;

    In relation to the specified template items, David Miall’s work was related to the following items:

    Content Expert, Authoring; Instructional Design; Internet Specialist

    3 Graduate student support breakdown:

    Juliette Klein $1778.42 for Summer, 1997 (12 hours, 2 months)

    Leanne Willson 5020.00 for Fall, 1997, Winter, 1998 (6 hours)

    Michael Snyder 5268.00 for Fall, 1997 (12 hours)

    4 Computer equipment and software was not originally budgeted as we anticipated using the facilities at the Production Studio of Academic Technologies for Learning. However, Don Kuiken was unable to use the Production Studio and requested a high-end computer for his development work. An additional computer and software were purchased to allow for development and evaluation in the department as we move into the second phase of the project.

    Equipment and Software Breakdown:

    Computer $4024.88 for Don Kuiken

    Front Page ’97 86.96

    Visual Basic Pro 143.23

    Front Page ’98 81.85

    Computer 2992.57 for departmental development and evaluation

    HomeSite 3.0 124.82

    MS Office 203.60

    Flash 3.0/Fireworks 312.05

    5 Travel breakdown:

    Don Kuiken, $96.34 prepare LEE II with Calgary

    Connie Varnhagen 379.14 observe intro. psych. classes, Ontario

    6 Miscellaneous Breakdown:

    Misc. computer supplies 323.00

    Duplication 148.31

    7 LEE I grant proposals did not require additional contributions. However, the Department of Psychology had committed to matching the ATL Fellowship funds awarded to Connie Varnhagen to provide additional teaching relief and graduate student assistance. The ATL Fellowship was turned down in lieu of receiving LEE I funding.

    8 The Faculty of Arts provided .5 course relief to David Miall, Department of English in Fall, 1997.

    9 The Department of Psychology provided 6 hours of support to Leanne Willson in Fall, 1997.

    10 The Department of Psychology has upgraded the departmental network to support offering of technologically enhanced courses. The specific cost cannot be estimated because this upgrade was also necessary to support research and departmental promotion.

    11 Although no financial value has been attached to the support received from Academic Technologies for Learning, staff at ATL and the Production Studio contributed to the following specified template items:

    Consultant(s), advisors; Instructional Design; Internet Specialist; Graphics/Interface Design; Staff Development; Evaluation

    Similarly, although no financial cost has been attached to the support received from the Department of Psychology, staff in the department contributed to the following specified template items:

    Consultant(s), advisors; Internet Specialist

    The Department of Psychology also contributed consultation, programming, and secretarial assistance.

    Back to Top of Page
    Back to Table of Contents

    Appendix A

    Sample Web pages:

    1. Home page for PSYCO 104 M1
    2. Syllabus for PSYCO 104 M1  Note: This link actually goes to the current syllabus for the intro.psych 104 class
    3. Home page for PSYCO 105 M1  Note:This link also goes to current intro.psych 105 home page
    4. Syllabus for PSYCO 105 M1  Note: Syllabus for current 105 section
    Back to Top of Page
    Back to Table of Contents

    Appendix B

    intro.psych brochure

    Back to Top of Page
    Back to Table of Contents

    Appendix C

    Sample Web pages:

    Reading the Research

    Back to Top of Page
    Back to Table of Contents

    Appendix D

    Sample Web pages:

    Literature and Psychology

    Back to Top of Page
    Back to Table of Contents

    Appendix E

    Objectives and Goals of the Undergraduate Curriculum in Psychology

    Objectives and Goals of the Undergraduate Curriculum in Psychology

    The mission of the University of Alberta is to enrich and enhance our community by disseminating knowledge through teaching and by discovering knowledge through research. Teaching and research are inextricably intertwined in the University, and graduate and undergraduate teaching efforts are influenced by both aspects of this broad mandate.

    The Mission of the Department of Psychology

    The Department of Psychology fulfills its part of the University mission by striving to achieve the highest standards of excellence in both teaching and research in the field of psychology. Although both graduate and undergraduate teaching efforts reflect research specializations within the Department, the Department is committed to undergraduate instruction in the discipline of psychology, broadly conceived.

    Objectives and Goals of the Undergraduate Curriculum

    In keeping with the mission of the University and the Department, the Department of Psychology adopts the following goals and objectives for the undergraduate program. These objectives may not be achieved in all cases or in all courses; moreover, many of the goals involve skills that may continue to develop over a student's lifetime. Nevertheless, the Department strives for these objectives and goals to be reflected in the structure and content of the program as a whole.

    The undergraduate curriculum in psychology is designed for students who seek majors or minors in psychology, as well as for those who seek acquaintance with psychology as part of their concentration in other areas. The curriculum involves a progressive focus on the acquisition of skills and practices of psychological research in general and an intensification in specialized study within various domains of psychology. The curriculum is adopted in cooperation with other departments and within the broad objectives of a liberal education in the arts and sciences.

    I. The Nature and Breadth of Psychological Studies

    Students enrolled in a program of studies in psychology are expected to:

    • acquire an understanding of and appreciation for the diversity of research practices and methods of inference within the discipline;
    • become conversant with the diversity of knowledge domains within the discipline, both currently and from an historical perspective;
    • and be able to articulate the relations between psychology and other disciplines in the arts and sciences.
    II. The Acquisition of Psychological Knowledge
    • Students enrolled in a program of study in psychology are expected to:
    • acquire an historical understanding of the social, cultural, and interdisciplinary influences on investigative practices in contemporary psychology;
    • become acquainted with the nature of psychological investigation, including its moral context, its logic and rationale, and the relations between evidence and explanation;
    • and be able to pose questions amenable to investigation, evaluate the design and execution of empirical research, and interpret research results in relation to theory and practice.
    III. Oral and Written Communication
    • Students enrolled in a program of studies in psychology are expected to:
    • acquire the oral and written skills necessary to articulate issues in psychological research and respond constructively to others' attempts to articulate those issues;
    • become critically evaluative, orally and in writing, of the research literature in psychology;
    • and be able to articulate, orally and in writing, well reasoned and coherent arguments in a style appropriate for academic or popular venues in psychology.
    IV. Applying Psychological Knowledge
    • Students enrolled in a program of studies in psychology are expected to:
    • acquire the skills necessary to relate their knowledge of psychology to personal, professional, and societal concerns;
    • become critically aware of the implications of psychological research for interventions designed to alleviate distress, enhance the well-being of others, or develop human potential;
    • and be able to understand the standards and responsibilities assumed by psychologists in their professional and scientific roles.
    Back to Top of Page
    Back to Table of Contents

    Appendix F

    Final Report of the Second Annual Teaching of Psychology Conference

    Second Annual Teaching of Psychology Conference

    June 19th-20th
    Grant MacEwan Community College
    Edmonton, Alberta
    Final Report
    Compiled by
    C.T. Scialfa
    University of Calgary

    Introduction

    In 1997, more than 40 teaching professors and publishers met at Red Deer College in Red Deer, Alberta to discuss issues related to the teaching of introductory psychology and the use of technology-based resources to facilitate that effort. By all accounts, this meeting was a stimulating success, and many of those attending felt it would be useful to have annual meetings devoted to issues in teaching academic psychology.

    The Second Annual Teaching of Psychology Conference (ATOP2) represented a continuation of this work. It was held on June 19-20, 1998 at the Grant MacEwan Community College in Edmonton. Peter Furstenau, Russell Powell and Andrew Howell served as our hosts, making arrangements for space, on-site meals and multimedia resources. Connie Varnhagen (University of Alberta) and Chip Scialfa (University of Calgary) organized the content of the program. LEE funds were used to provide financial support for conference participants and were also the major source of support for many of the projects discussed at the conference.

    The meeting was divided into several symposia and workshops. Some were targeted at the goals of our collaborative LEE-funded project to design and evaluate technology-based courseware for introductory psychology. Included in this category were sessions on defining the content of the course, assessing difficulty of that content, characterizing commercially available software, Web development issues, and determining the design and evaluation strategies for next year's phase of the project. In addition, there was a symposium on general issues in the teaching of psychology, with a keynote address by Dr. John Mitterer of Brock University. Thus, the conference was stimulating and valuable for those directly involved in teaching introductory psychology, but also for those people who were interested in the more general issues surrounding teaching of psychology in the Information Age.

    ATOP3 is tentatively scheduled for May, 1999 at Augustana College. If you are interested in getting more information about ATOP2, ATOP3, or other LEE-funded projects in psychology, contact Connie Varnhagen (varn@psych.ulberta.ca) or Chip Scialfa (scialfa@acs.ucalgary.ca). If you'd like to be added to our discussion list, contact Gary Finley (gfin@psych.ualberta.ca) and let him know that you'd like to be added to the INTROP listserver.

    Conference Schedule

    FRIDAY, JUNE 19TH

    1:00 pm- Opening Remarks (Peter Furstenau, Chip Scialfa and Connie Varnagen)

    1:30 pm

    1:30 pm- Keynote address (John Mitterer, Brock University)

    2:30 pm Title: Evaluating Educational Software

    2:30 pm- BREAK

    2:45

    2:45 pm - Teaching of Psychology Symposium

    5:45 pm (Coordinator: Connie Varnhagen, University of Alberta)

    Connie Varnhagen (University of

    Alberta) and Nancy Digdon (Grant MacEwan Community

    College).

    Title: Reading the Research: Helping Students Learn How to Read Empirical Research Reports

    Jeff Caird and Brad Johnson (Univerisity of Calgary)

    Title: Ten Lessons Learned from the Integration of Web Pages and Chat Forums into Upper Level Psychology Courses

    Scott Oddie (Red Deer College)

    Title: Context-based Learning of Introductory Psychology

    Connie Varnhagen and Leanne Willson (University of Alberta)

    Title: Using the Internet to Enhance Critical Thinking in Introductory Psychology.

    Chip Scialfa, Stacey Nairn, and John Ellard (University of

    Calgary)

    Title: Content Analysis of Introductory Psychology Texts

    SATURDAY, JUNE 20TH

    8:30 am- Mapping the Territory: The Content of Introductory

    9:30 am Psychology (Coordinator: Chip Scialfa, University of Calgary)

    1. Past content analyses of texts (Chip Scialfa, U of C)

    2. Content analysis and concept mapping

    (Chip Scialfa & Brad Johnson, U of C)

    3. Student understanding of text material

    (Chip Scialfa and Brad Johnson, U of C)

    4. Discussion

    9:30 am- What They Get and What They Don't: Differential

    10:30 am Understanding of the Content of Introductory Psychology

    (Coordinator: Louis Dingley, Keyano College)

    1. Retesting in a Computer-Managed Learning course

    (Russ Powell, GMCC)

    2. What can the GRE tell us about concept difficulty

    (Louis Dingley, Keyano College)

    3. Item analysis of examinations

    (Larry Wenger, Connie Legare and Chip Scialfa, U of C)

    4. Student assessments, objective and subjective

    (Larry Wenger, Connie Legare and Chip Scialfa, U of C)

    10:30 am What's Out There: Characterizing Commercial Applications

    Noon ( Bob Heller, Concordia University College)

    Longman Psychology CD Rom Worth Psychology Multimedia Exhibit

    PsychQuest PsykTrek

    The Integrator (Biological Psycholo The Integrator

    Digital Resource Library Multimedia Psych

    Noon - Lunch at GMCC facilities

    1 pm

    1:00 pm Web Development Issues

    2:00 pm (Coordinator: Connie Varnhagen, University of Alberta)

    1. Request for Web Needs

    2:00 pm- Research Design and Evaluation

    5:00 pm

    (Coordinator : Scott , Red Deer College)

    1. Getting adequate controls

    2. Matching applications to pedagogical goals

    3. Assessment - The How To Issue

    4. Assessment - Implications for teaching

    5. Discussion

    Abstracts

    John Mitterer (Brock University) - "Developing and Evaluating Educational Software."

    After congratulating the conference participants on past and current efforts, Dr. Mitterer pointed out that collaborations on this scale are unprecedented, even in the larger population centers of North America. He discussed some of the forces that render these efforts particularly timely, among them the need for research on the application of technology to education, improvement in that technology, the need to overcome instructional frustrations associated with large classes, static images, and a desire to understand how students interact with knowledge to master it. He suggested that an alliance of students, instructors, multimedia authors, evaluators, post-secondary administrators, governments, and publishers can be an economically viable way of driving the development of instructional environments of the future. Dr. Mitterer discussed several specific threats and concerns related to our efforts including the following:
    - Spreading ourselves too thinly in development or evaluation will be expensive and counterproductive.
    - Splitting into factions will not work. Larger numbers are essential.
    - It is unlikely that several publishers will cooperate with each other in this effort.
    - Evaluation is difficult but necessary. The problem space is too large to be manageable without experimental controls. The evaluation must distinguish between formal variables (e.g., screen size) and content variables (e.g., extent of writing). It may be more important to emphasize formative, not summative evaluation so as to improve the next generation of resources. There is also a need to distinguish between content and process, and to match evaluation to the purpose of the software.

    Connie Varnhagen (University of Alberta) and Nancy Digdon (Grant MacEwan Community College) - "Reading the Research: Helping Students Learn How to Read Empirical Research Reports."

    The objectives of this Internet-based module are to help students learn how to read published reports of empirical research. Reading the Research consists of a brief introduction to the genre of the report style, describes critical thinking questions for reading a report of empirical research, and provides guided experience in reading a published report. The instructor writes a summary and integrates two types of critical thinking questions into the summary. Factual questions help students identify important information from the report. Students determine their understanding of the key information in the report by answering self-test questions that provide immediate feedback as they read the summary. Discussion questions help students evaluate the importance, relevance, and generalizability of the research. Students discuss these questions among themselves in computer conferencing discussion groups. A formative evaluation of Reading the Research was conducted with an experimental psychology class at Grant MacEwan Community College. Students expressed positive attitudes about using the module, felt they had learned about the structure and content of the research report, and felt that Reading the Research should be used in other classes. The module will be part of experimental introductory psychology courses at the University of Alberta and will also be used in other courses at Grant MacEwan Community College. We encourage other instructors to use and evaluate Reading the Research in their classes as well.

    Jeff Caird and Brad Johnson (Univerisity of Calgary) - "Ten Lessons Learned from the Integration of Web Pages and Chat Forums into Upper Level Psychology Courses".

    In order to best leverage the funding provided by year 1 LEE funding, it was determined that the best use of both expertise and funds was in the generation, integration, and fine tuning of a series of web-based forums or discussion groups. Server-side technology was used which effectively increased both access and usability while a series of questions provided content for the forums. Lessons learned from the project fell into roughly three categories: resource lessons, pedagogical lessons, and more generally, how best to make use of the forums e.g. how to form questions to generate discussion. No formal analysis was conducted as the major part of this project was to determine both how best to construct discussion tools given a large set of access and usability constraints, and how best to use the forum once it was constructed. A logical next step is to begin a more formal process of testing and evaluation to determine whether any performance gains may be generated through use of forums or discussion groups.

    Scott Oddie (Red Deer College) - "Context-based Learning of Introductory Psychology".

    Traditional (TRAD) and context based learning (CBL) paradigms use different teaching philosophies to deliver introductory psychology curriculum at Red Deer College (RDC). In a TRAD lecture course, learners are exposed to the course objectives and pertinent course curriculum in class as presented by an instructor. A text, videos, in class demonstrations, in class discussions, and group exercises are used to aid learners in the mastery of course objectives. In contrast, CBL proposes to relate curriculum to "real-life" scenarios which are used to introduce and outline related introductory psychology objectives. From the scenario, learners identify in groups what they currently know and may need to know. CBL resources are similar to those listed for TRAD and include an expert in the field of psychology. In contrast to TRAD, the CBL curriculum is presented in an unstructured manner. That is, CBL learners must examine and master the breadth of the course curriculum using predominantly their own guidance and skill.

    We examined whether these different teaching philosophies resulted in inconsistent delivery of course content and academic performance of learners enrolled in TRAD or CBL introductory psychology courses at RDC. It was determined that CBL learners examined fewer objectives (and concepts), and spent more time on the course than TRAD learners but the academic performance of TRAD learners did not differ significantly from those in the CBL group.

    Connie Varnhagen and Leanne Willson (University of Alberta) - "Using the Internet to Enhance Critical Thinking in Introductory Psychology".

    The goals of most introductory psychology courses include developing critical, analytical, and integrative thought about psychological principles and processes. Given large lecture classes and little support for laboratory exploration, these goals are seldom met. At the University of Alberta, we are developing intro.psych, an Internet supplement for introductory psychology courses that is designed to help students develop essential critical thinking skills. Components of intro.psych include critical thinking exercises, called Sites to Visit, that make use of the World Wide Web (WWW) and computer conferencing.

    A formative evaluation of these exercises was conducted with an Intersession class in introductory psychology. Students appeared to benefit from the exercises; they enjoyed the real life examples they encountered on the WWW, had a positive impression of the supplement, and learned computer literacy skills from the experience. As well, they preferred the format of less lecture and use of the Internet supplement to a traditional lecture format. We will be expanding our use of intro.psych in experimental, technologically enhanced courses in the Fall and Winter sessions, 1998-1999.

    Chip Scialfa, Stacey Nairn, and John Ellard (University of Calgary) - "Content Analysis of Introductory Psychology Texts".

    Six of the best selling introductory psychology texts were examined to uncover the concepts treated in the chapters on sensation and perception. The total list consisted of almost 800 concepts, with approximately 75 concepts being covered by at least 5 of the 6 texts. This approach, once expanded to include the entire text, can provide the conceptual core of introductory psychology and can be used to assess student-based and instructor-based assessment of conceptual difficulty.

    Session Summaries

    Mapping the Territory: The Content of Introductory Psychology (Coordinator: Chip Scialfa, University of Calgary).

    Following an historical overview of page count and concept list approaches to delineating the core concepts in introductory psychology. Dr. Scialfa described recent efforts at the University of Calgary to find common concepts in chapters on sensation and perception from the six best-selling introductory texts in Canada. He then discussed contemporary research on concept mapping and summarized pilot work with Brad Johnson in assessing faculty responses to the use of concept mapping. Discussion centered on using concept maps in an experimental section of introductory psychology and how technology-based mapping tools might be used and evaluated.

    What They Get and What They Don't: Differential Understanding of the Content of Introductory Psychology (Coordinator: Louis Dingley, Keyano College).

    Dr. Dingley initially described his analysis of the item difficulty of questions on exams from his introductory psychology courses. While differences across topics were not large, students performed best on the topics of perception, consciousness, learning and memory, cognitive development, social development, motivation and health. They performed worst on questions pertaining to history, language, thinking, social psychology, personality and therapy. Dr. Powell then described his efforts to examine content difficulty in his computer-managed learning (CML) introductory psychology course. Students had to take retests most frequently on the chapters on learning, motivation and emotion. There were no differences in retest frequency for the chapters on statistics, neuroscience, sensation and perception, consciousness, and memory. Louis Dingley then reported that, while the ETS has some data on the relative difficulty of material on the GRE-Advanced in psychology, it was too coarsely analysed to be of value. In addition, ETS asserted that there are too many moderating variables to make general statements about relative difficulty. Next, Dr. Scialfa discussed analyses of more than 1000 test items carried out by Larry Wenger and Connie Legare at the University of Calgary. Results suggest small differences in difficulty across topics. Finally, Dr. Scialfa presented a questionnaire that might be used by students to assess the relative difficulty of topics in the course. It was suggested that this approach allows for a test of the validity of the item analysis technique and might also dovetail nicely with concept-mapping studies currently under way.

    What's Out There: Characterizing Commercial Applications ( Bob Heller, Concordia University College).

    Dr. Heller described the system he developed to describe a large number of software products designed for use in the teaching of psychology. This extensive effort is summarized below. In concluding, Dr. Heller focused on the following:
    - Products will involve increased hardware requirements.
    - There is a good deal of among-product variance in testing modules, glossaries, multi-media effectiveness, Internet connectivity, cost and accessibility.
    - There is limited use of stated objectives and so, little connection between the products and pedagogical objectives.
    - There is next to no progress monitoring.
    - There is next to no emphasis on writing.
    - There is little effort to develop critical thinking.
    - There IS emphasis on psychological content.

    Brief Descriptions of Selected Commercial Software

    1. CyberPsych by John Wiley & Sons

    CyberPsych is an electronic study guide designed to accompany ‘Psychology: Mind, Brain, & Culture’ by Drew Westin. The primary organization is by chapter. Each chapter is reviewed by entering a room and clicking on various objects to enable quizzes, exercises, glossaries, case studies and videos (when available).

    2. The Core on CD ROM by Harcourt Brace

    The Core is an electronic supplement to the text ‘In Search of the Human Mind’ by Robert Sternberg. It consists of a full text version of the Sternberg text that is accessible (via hyperlink) from different components in The Core. The Core also consists of Conceptual Overviews, minilectures, and quizzes that are organized by chapter. It also features an on-line tour of the disk.

    3. Multimedia Psych by Prentice Hall

    Multimedia Psych is a an electronic study guide that supplements the text ‘Psychology’ by Morris. It consists of four components, three of which are relatively stand alone (Games, Study Guide, and Animations) and one (Interactive Book) that requires in-house software (Frame Viewer) in order to operate. All components, with the exception of Animations are organized by chapter. The Interactive Book contains a multimedia Overview, a hyperlinked Index, and a Table of Contents. The Table of Contents provides access to the different chapters. Each chapter contains a Quick Tour in the form of an audio overview, a set of learning objectives with hyperlinks to the full text, and a set of general review questions. The bulk of the chapter is accessed through a Full Text version or through an electronic TA (an animated figure with audio that provides a brief overview of the chapter). Both means of access contain essentially the same content (text, graphics, & videos).

    4. PsychQuest: Interactive Exercises for Psychology (Demo version) by Worth

    PsychQuest is a set of 8 modules that examine high interest research topics that relate psychology to real world issues. Each module provides (a) background information (predominately in text form) with diagrams and click on definitions, (b) activities and demonstrations supplemented with video, animation, & audio, (c) references for further reading, (d) hot weblinks, and (e) a 20 item multiple choice test. It is text independent.

    5. Psychology CD ROM (Prototype) by Longman

    Psychology CD ROM is a product under completion. The product is organized by Topic (which generally correspond to broad chapter headings) and by Theme (which is the same content in Topic but organized thematically). Under each topic are several subtopics that are described an Introduction section, illustrated with 1-2 activities/demonstrations, and accompanied by a set of detailed references and a 10 question multiple choice exam. Each subtopic also has a control bar that enables access to More Information, Glossary, Weblinks, Notepad, & Help. (Note: very little was functional in the prototype).

    6. PsycTrek: A Multimedia Introduction to Psychology by Brooks/Cole

    PsycTrek is an full electronic study guide and electronic supplement designed to accompany Psychology: Themes and Variations by Wayne Weiten. The product consists of a set of 12 learning modules that roughly correspond to the chapters in Weiten, a study guide (complete with learning objectives, concept review, person review and quizzes), 7 simulations, and a glossary. The learning modules make use of audio, animation, and video. The modules also have embedded review checks.

    7. PsychSim: Interactive Graphic Simulations for Psychology (Version 4) by Worth

    PsychSim is an electronic supplement designed to accompany any (Worth) psychology text. The product consists of 19 modules that are designed to either simulate difficult-to-illustrate psychological processes (e.g. neural impulses) or demonstrate classic psychological experiments (e.g. Sperling’s partial report procedure). In addition to the simulations/experiments, each modules consist of approximately 40 pages of accompanying text and graphics. The modules conclude with a summary, references and a glossary. It comes on floppies or CD ROM.

    8. The Integrator by Brooks/Cole

    The Integrator is an electronic supplement to Brooks/Cole books (Weiten; Plotnik; Kalat & Nairne) for students and a lecture-maker resource for instructors. Both functions make use of the same digital resources. The student supplement is organized by chapter and includes essentially one or more of the following types content presentations: (a) slide shows (diagrams that can be optionally accompanied with text and/or music) (b) activities (may involve student participation), (c) study pages (which are glossaries) and (c) thoughts on … (quotations from famous individuals).

    Research Design and Evaluation (Coordinator : Scott Oddie, Red Deer College)

    Several hours were spent Saturday afternoon discussing issues related to carrying out the evaluations of technology-based courseware. The discussions fell into several broad categories including the delineation of pedagogical goals, use and development of adequate controls, and issues of design. Following is a summary of those discussions.

    Goals

    - Increase depth of treatment of material
    - Increase methodological understanding
    - Increase understanding of psychology as science
    - Improve ease of reading peer-review literature and summarizing same
    - Build writing skills.
    - Improve ability to work in groups.
    - Engage students in the "stuff" of psychology Controls and Confounds
    - Motivation. Perhaps adopt a waiting-list control approach so that motivation in control group and treatment groups are equated. Alternatively, one might use a repeated-measures design.
    - Computer literacy. If students are giving reactions to computer applications, these must be done after they have gotten beyond the steepest portion of the learning curve.
    - Instructor interest and motivation important. Pick the best instructors or match for teaching ratings for both treatment and control conditions.
    - Hawthorne effect. Perhaps assess differential impact over time since Hawthorne effect should dissipate.
    - Must document the ease of use. If treatment does not "work" (e.g., frequently crashes) then it may not be used or viewed favourably.
    - Verifying use is critical to control for time on task, among other things.
    - Pre-assessment of prior exposure to computers, interest level in material, GPA, etc.

    Design Issues
    - Scope. Target manageable topics or modules and make comparisons at a low enough level of complexity to allow for reasonably unambiguous inferences about the processes involved.
    - Gather multiple measures over time.
    - Construct theoretically meaningful comparisons.
    - Calibrate outcome measures with the intended purpose of the software.
    - Measures should include depth of processing, time on task, interest, footprints, affective reactions, comprehension, short-term retention, long-term retention, transfer, and writing skills
    - Be wary of cohort and subject effects.
    - Possible products and tools for evaluation: concept mapping, Reading the Research, Cyber Rat, Psych Trek, discussion groups and chat rooms, OCR and Acrobat, simulations gathered from the group and the Web.

    Analysis and Conclusions

    ATOP2 was successful in (1) bringing together teaching psychologists who share the goal of improving the educational experience of students involved in introductory psychology, (2) communicating within the group the results of various efforts to enhance teaching, (3) delineating more precisely the design and evaluation issues in future work and (4) exposing the group to some of the courseware that can be evaluated in the coming year. In this latter regard, participants expressed an interest in pursuing more work in the following areas: difficulty analysis, concept listing, concept-mapping, discussion/chat groups, Reading the Research, intro.psych, and Psych Trek. Bearing in mind that an evaluation of even a portion of this set is not possible without continued LEE funding, the next step is to formalize partnerships and begin pilot work on these collaborative projects or continue the collaborative work already begun.

    Defining the Core Concepts in Introductory Psychology Texts

    This would require following an agreed upon protocol for outlining the texts, aggregating the data and evaluating questionable concept entries (e.g., potential synonyms). Chapters on sensation and perception and social psychology have already been completed. Those indicating an interest in partnering on this project include Larry Wenger, John Ellard, Connie Legare, Chip Scialfa (University of Calgary), Louis Dingley (Keyano College), and Tim Parker (Augustana College).

    Difficulty Analysis I

    This would require getting item analyses on examinations following Webb's (1991) chapter taxonomy as has been done for approximately 1000 items to date. The results would then be aggregated at the University of Calgary. Those indicating an interest in partnering on this project include Larry Wenger, John Ellard, Connie Legare, Chip Scialfa (University of Calgary), Louis Dingley (Keyano College), and Scott Oddie (Red Deer College).

    Difficulty Analysis II

    This would require developing further a questionnaire to assess student ratings of topic difficulty. It would be done at a global level, following Webb's (1991) chapter taxonomy, but also at a more fine-grained level using the concept lists generated from

    the project on defining the core concepts in psychology. Those indicating an interest include Larry Wenger, John Ellard, Connie Legare, Chip Scialfa (University of Calgary), and Louis Dingley (Keyano College).

    Concept Mapping

    Those indicating an interest include Brad Johnson, John Ellard, Chip Scialfa (University of Calgary), and Louis Dingley (Keyano College) and Karsten Loepelman (University of Alberta). Details to be developed, but Brad Johnson and Chip Scialfa (University of Calgary) are piloting a concept mapping exercise in a Summer, 1998 session of introductory psychology.

    Writing

    Those indicating an interest include Brad Johnson, John Ellard, Chip Scialfa (University of Calgary), and Louis Dingley (Keyano College). Details to be developed, but Brad Johnson and Chip Scialfa (University of Calgary) are piloting a writing exercise in a Summer, 1998 session of introductory psychology.

    Discussion Groups

    Those indicating an interest include Brad Johnson, Chip Scialfa (University of Calgary), and Louis Dingley (Keyano College). Details to be developed but Brad Johnson (University of Calgary) is the contact person.

    Web Development and Evaluation

    Reading the Research, a module designed to help students learn how to read and critically evaluate empirical research, will be used and evaluated by a number of ATOP faculty, including Nancy Digdon (GMCC), Connie Varnhagen (University of Alberta), Louis Dingley (Keyano College). It will also be used at the Harcourt Brace Website, maintained by John Mitterer. The Larger Web-based introductory psychology supplement, intro.psych, will be evaluated in experimental sections of introductory psychology taught by Copnnie Varnhagen and Don Heth (University of Alberta). Continued Web-based psychology supplement development and evaluation is directed by Connie Varnhagen and Don Heth based on input and suggestions by any ATOP member.

    PsychTrek

    Those indicating an interest include Louis Dingley (Keyano College) and Andrew Howell (GMCC). Details to be worked out but Andrew Howell (GMCC) is the contact person.

    Participants

    Name   Institution
    Larry Andreotti   Grand Prairie Regional College
    Charles Beck   University of Alberta
    Patricia Boechler   University of Alberta
    John Connors   Canadian University College
    Louis Dingley   Keyano College
    John Ellard   University of Calgary
    Peter Fustenau   Grant MacEwan Community College
    Darren George   Canadian University College
    Bob Heller   Concordia University College
    Andrew Howell   Grant MacEwan Community College
    Brad Johnson   University of Calgary
    Karsten Loepelmann   Grant MacEwan Community College
    Heather Looy   Kings College
    Paula Marentette   Augustana College
    John Mitterer   Brock University
    Scott Odie   Red Deer College
    Tim Parker   Augustana Collge
    Russell Powell   Grant MacEwan Community College
    Chip Scialfa   University of Calgary
    Connie Varnhagen   University of Alberta
    Leanne Willson   University of Alberta
    Larry Wenger   University of Calgary

    Back to Top of Page
    Back to Table of Contents

    Appendix G

    Financial Reports:

    1. March, 1998

    2. September, 1998

    Back to Top of Page
    Back to Table of Contents