Portfolio Assessment

• assigned homework problems from each section of the course
• additional completed problems, including one non- trivial problem selected by the student from each chapter covered
• oral presentations support materials
• other work as assigned

The portfolios undergo a peer review and an instructor review near or at the end of each chapter. The standards for evaluating the portfolio contents include:

• concise and well-written materials, including the use of complete sentences to explain the work and the outcomes.
• good attempt at solving the problems, which includes showing appropriate thought and reasoning even though a final resolution may not have been achieved.
• valid results which follow from the work.

At the beginning of the semester, students receive the initial set of rules for preparing their portfolios. The rules and criteria for evaluation are included in the syllabus. Students are also given an additional handout with specific directions as to what is to be included in the portfolio. Students are asked to include not only homework problems or assignments, but extra things they produce which highlight their strengths and abilities. For example, they can choose to solve one extra problem from each chapter that meet our conditions: problems have to be non-trivial. If there are any doubts about the appropriateness of a problem, students are to check with us before investing time working on it.

The purposes of the writing component include:

1. Writing does assist the learning process, "writing for learning." Writing helps clarify concepts for the students; helps the students organize, understand, make connections, clarify processes and solutions; helps the students understand the assumptions made about the solutions to problems; helps the students interpret the results of their work.
2. Writing is an important element in the students' future academic and professional life.

Our students are also expected to plan and deliver an oral presentation covering the content for one of the chapters. All materials prepared for presentation (topic outline, visuals, handouts, etc.) must be included in the portfolio as well.

Another component of the portfolio is the comments from peer review and students' response to those comments. The concept is to give students a chance to change things, to improve them before the final grade. Students exchange portfolios before the instructors see them. Sometimes we assign the peer reviewers and sometimes we let them choose their own. The peer reviewer comments are included in the portfolio and the reviewers is graded on their comments. We examine the student work to see if s/he has responded (changed their work) as a result of the review. We believe that both students learn from the peer review process. It has been as meaningful to the reviewer as to the one being reviewed.

Portfolios are evaluated primarily on content. All students receive an evaluation sheet with written comments. If students have any questions or concerns about the comments, they are encouraged to discuss their concerns with one of the instructors. This semester we assigned a score to each section of the portfolio completed and reviewed. We did not think that was an ideal strategy since some students felt that because they had completed the section and received a score, they did not have to deal with it anymore. We tried to change that attitude by telling students that after they received feedback from us, they could still improve their work and modify their score by changing the portfolio and adding to it.

One of our motivations for incorporating portfolios into the course was to experiment with alternative assessment. We also believe that students should be able to collect their best materials and put them in a portfolio that they could carry to an employer to show. It is part of professional growth. It has been difficult, however, to make students understand the idea and to make them see the value of collecting samples of their work to show change and growth.

We were hoping that students would on a regular basis contribute to their portfolios, get feedback about the portfolio, and make improvements based on the feedback. At one point we thought that we would do very informal evaluations several times during the semester and then the final evaluation at the end of the term. We realized early on that students were not putting anything on the portfolio; they were not keeping the portfolio up-to-date, so we changed our thought process and expectations and imposed some portfolios deadline.

Portfolio is a new concept for both students and instructors. However, we still believe in the value of portfolio as an assessment tool and as a learning tool, and would do it again with third semester calculus students.

Guidelines for Test Construction

STEP 1

STEP 2

STEP 3

• Match the scoring criteria to the type of assessment measure.
• Identify and define quality performance.
• Gather samples of student's work that exhibit a range of performances.
• Discuss with peers the characteristics between effective and ineffective product, solution, performance, etc. Criteria should hold up to several judges looking at the assessment measure and reaching the same conclusion about the learner's attainment of the objectives.

Constructing Multiple-Choice Test Item

• Match the test item to the learning objective.
• Each multiple-choice item should include a stem and 3-4 plausible distracters. The stem should present a single, definite statement of a problem; should always include a verb in the statement; and should present the problem in a concise, clear, simple manner.
• The stem should include only the necessary information to understand the problem. Avoid irrelevant, trivial, and obvious material. Avoid unnecessary repetition of material in the options by including as much of the item as possible in the stem.
• Ensure that all the distracters are plausible and that only one response is correct or the best answer.
• Be sure each item is independent of all others. A question should not provide a clue to another question later in the test.
• Use positive statements as much as possible. If negative statements must be used, then CAPITALIZE, underline, or highlight the negative term.
• Arrange the options in logical order (by number, alphabetically, chronologically), if possible. If there is no logical order, randomize.
• Avoid copying statements verbatim from the text when designing stems and options.
• Write options in grammatically parallel form whenever possible.
• Avoid using options such as "all of the above" and "none of the above."
• Avoid providing clues to the correct option:
  • Length: do not make the correct choice the longest option.
  • Verbal association: do not include different forms of the same word in both the stem and the correct option.
  • Grammatical: all the options must complete a grammatically correct statement. Beware of the use of "a," "an," plural forms, and gender forms.
  • Determiners: beware -- "all," "never," and "always" are usually associated with the incorrect responses. "Usually," "sometimes," "typically," and "maybe" are usually related to correct options.