Assessment: an ongoing process aimed at understanding and improving student learning. It involves: making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance.

Backload (--ed, --ing): amount of effort required after the data collection.
Competency: level at which performance is acceptable.
Convergent validity: general agreement among ratings, gathered independently of one another, where measures should be theoretically related.

Criterion-referenced: criterion-referenced tests determine what test takers can do and what they know, not how they compare to others. Criterion-referenced tests report how well students are doing relative to a pre-determined performance level on a specified set of educational goals or outcomes included in the curriculum.

Externality: the extent to which the results of the assessment can be generalized to a similar context.

External validity: the extent to which the results of a study are generalizable or transferable to other settings. Generalizability is the extent to which assessment findings and conclusions from a study conducted on a sample population can be applied to the population at large. Transferability is the ability to apply the findings in one context to another similar context.

Forced-choice: the respondent only has a choice among given responses (e.g., very poor, poor, fair, good, very good).

Formative assessment: intended to assess ongoing program/project activity and provide information to improve the project. Assessment feedback is short term in duration.

Frontload (--ed, --ing): amount of effort required in the early stage of assessment method development or data collection.

Generalization (generalizability): the extent to which assessment findings and conclusions from a study conducted on a sample population can be applied to the population at large.

Goal-free evaluation: focuses on actual outcomes rather than intended program outcomes. Evaluation is done without prior knowledge of the goals of the program.

Inter-rater reliability: the degree to which different raters/observers give consistent estimates of the same phenomenon.

Internal validity: (1) the rigor with which the study was conducted (e.g., the study's design, the care taken to conduct measurements, and decisions concerning what was and wasn't measured) and, (2) the extent to which the designers of a study have taken into account alternative explanations for any causal relationships they explore.

Longitudinal studies: data collected from the same population at different points in time.

Norm (--ative): a set standard of development or achievement usually derived from the average or median achievement of a large group.

Norm-reference: a norm-referenced test is designed to highlight achievement differences between and among students to produce a dependable rank order of students across a continuum of achievement from high achievers to low achievers.

Observer effect: the degree to which the assessment results are affected by the presence of an observer.

Open-ended: assessment questions that are designed to permit spontaneous and unguided responses.

Operational (--ize): defining a term or object so that it can be measured. Generally states the operations or procedures used that distinguish it from others.

Reliability: the extent to which an experiment, test, or any measuring procedure yields the same result on repeated trials

Rubrics: a set of categories that define and describe the important components of the work being completed, critiqued, or assessed. Each category contains a gradation of levels of completion or competence with a score assigned to each level and a clear description of what criteria need to be met to attain the score at each level.

Salience: a striking point or feature.

Stakeholder: anyone who has a vested interest in the outcome of the program/project.

Summative assessment: assessment that is done at the conclusion of a course or some larger instructional period (e.g., at the end of the program). The purpose is to determine success or to what extent the program/project/course met its goals.

Third party: person(s) other than those directly involved in the educational process (e.g., employers, parents, consultants)

Triangulate (triangulation): the use of a combination of assessment methods in a study. An example of triangulation would be an assessment that incorporated surveys, interviews, and observations.

Topology: mapping of the relationships among subjects.

Utility: usefulness of assessment results.

Variable (variability): observable characteristics that vary among individuals responses.

Validity: the degree to which a study accurately reflects or assesses the specific concept that the researcher is attempting to measure. Validity has three components:

relevance - the option measures your educational objective as directly as possible

accuracy - the option measures your educational objective as precisely as possible

utility - the option provides formative and summative results with clear implications for educational program evaluation and improvement

Used with permission from Gloria M. Rogers, Vice President for Institutional Research, Planning, and Assessment, Rose-Hulman Institute of Technology