Objectives-based assessment

Alterantive Grading

This course uses an alternative grading system based on demonstrating mastery of course learning objectives. The system is designed to be flexible and to allow students to demonstrate their skills in a variety of ways. This page provides an overview of the assessment system, including the learning objectives and standards that students are expected to meet.

If you’re new to alternative grading systems, this might be a bit confusing at first. I encourage you to read through the whole page and then reach out with any questions you have.

A couple things to note about how I talk about grading in this course:

  1. I will often refer to the skills you are expected to demonstrate as “standards” or “objectives” interchangeably.(You are graded by the standards of meeting the course learning objectives.)
  2. For convenience, I tend to call this a “standards-based” grading system, but it is more a hybrid of several alternative grading systems (mostly a blend of standards- and specifications-based systems). The main idea is that you are graded based on what you can do, not how you got there.
  3. “Engagement” is a separate component of your grade that is not strictly based on the standards, but still reflects broader learning objectives of the class. Although the engagement component of your grade is more subjective than the standards-based component, it is still based on the learning objectives and is intended to be fair and transparent. One major objective of the class is to get you to care about what you’re doing. Another is to get you comfortable with the process of learning and problem-solving, especially collaboratively.

Why do things have to be so complicated?!?!

I know, I know. I’m sorry. I have a few reasons for using this system:

  1. Transparency: I believe it is a more fair and transparent way to grade. It allows you to focus on learning and improving your skills rather than on getting a certain grade.
  2. Flexibility: It also allows you to demonstrate your skills in a variety of ways, which can be especially helpful if you have a particular strength or interest. It lets you focus on what you actually want and need from the class. I have things I want you to learn, but I also want you to have the freedom to explore and learn things that are important to you and that will allow to you make progress on a project you care about.
  3. Efficiency: Grading from the assessment checklist is much faster and more consistent than traditional grading. This means you get feedback more quickly and the TAs and instructor can spend more time helping you learn and less time grading. You typically won’t get many (or sometimes any) written comments on your work, because the assessment checklist should tell you everything you need to know. (Of course, if you have questions or want more feedback, you can always ask!)
  4. Autonomy: You’re adults, and I believe you should have control over your priorities. With this system, you get to decide which areas to focus on, which times of the quarter you want to put in the most work, and even whether you care about the grade at all.
  5. Equity: Above all, alternative based grading systems are designed to be equitable. It reduces subjectivity. It minimizes effects of implicit bias. It allows multiple graders to evaluate consistently. It does not punish students who learn at different speeds or take different paths to achieve the same mastery. It gives all students multiple opportunities to understand what is expected of them and adjust their work accordingly. Your grade should reflect your skills and effort, not your background or your ability to guess what the instructor wants.

If you’re interested in learning more about alternative grading practices or the research behind them, I’m happy to chat about that too! Here are a few resources to get you started:

  1. Grading for Growth (Clark & Talbert, 2023) - available as a free eBook from UChicago library
  2. Beyond “the Grade”: Alternative Approaches to Assessment - Harvard Center for Teaching and Learning
  3. Alternative Grading Approaches

Assessment Overview

Final course grades are assigned based on students’ demonstration of 30 standards based on the course learning objectives.

Mini-projects and the final research project each have an engagement component worth 10 points. This score is based on the investment you demonstrate in learning the course material separate from meeting learning objectives.

This page gives detailed information about the standards-based grading aspects of the course. Refer to the grading policies for information about how grades are calculated for both the objectives-based and engagement components.

Research project

tl;dr: Research projects should check off all 30 unique standards. Drafts can be submitted up to 4 times throughout the quarter, with the final grade being that of the final submission.

Students’ research projects should demonstrate all 30 standards. Students may submit working drafts of their projects up to 4 times throughout the quarter (max 1 per week). The project grade will reflect the standards met in the final submission, which may be any time during the quarter.

  • There are very high expectations for the research project; meeting each standard equates to 1 point of your final course grade. Each draft submission is an opportunity to see which standards you have met so far and to add or revise as needed.
  • Drafts take time to grade, especially as the quarter progresses and I receive more submissions each week. Anticipate a turnaround time of about a week, though it will likely be quicker earlier in the quarter.
  • Only the final submission needs to be a complete draft. I encourage you to submit incremental drafts as we progress through the syllabus. For example, your first draft might just demonstrate GitHub and Markdown skills, your second adds basic programming and data wrangling, your third adds visualization and analysis, and your fourth is a complete draft correcting previous mistakes.
  • In theory, any attempt can be your final submission at any point in the quarter. In other words, if you’re a real overachiever and work ahead to produce a full draft that meets all 30 standards in Week 5, you’re done! Or, you may decide that the draft that met 20 standards is actually good enough to meet your personal goals for the class and your final grade and opt not to submit additional attempts.

Mini-projects

tl;dr: Mini-projects should demonstrate at least 20 unique standards from the assessed learning objectives. Another 20 points can be earned by demonstrating a combination of assessed standards and/or unassessed learning objectives. You cannot demonstrate the same standard more than one per assignment. You can submit as many or as few mini-projects as you like, but you cannot resubmit the same mini-project. You can select mini-projects from the assessment menu or design your own (with instructor approval).

Students should demonstrate mastery of the standards across at least assessed “mini-projects” supplemental to their research project. Student may choose from the “menu” of assessments and/or propose their own.

  • Along with each assessment you will submit a checklist of the standards, marking off each standard you are attempting to demonstrate.
  • Across all assessments, you should demonstrate at least 20 unique assessed standards (20 points).
  • An additional 20 points may be earned by:
    • demonstrating some combination of 20 assessed standards (not necessarily unique) and/or unassessed learning objectives
      • The list of “unassessed learning objectives” includes things I hope you will come away from the class having learned but that are not necessarily practical skills I expect you to demonstrate. Showing that you have met these objectives might not easily take the form of a technical, applied project. The menu has several options for this approach, but you are also welcome to design your own.
    • completing an “off-the-syllabus” project
      • If you’re feeling ambitious and/or bored with the course material, you can fulfill some or all of those 20 points with an “off-the-syllabus” (OtS) project. In this option, you’ll take the initiative to independently learn a few skills beyond the scope of this class (they should still be clearly related to the course!!). The menu has several options for OtS projects, but I encourage you to tailor this to your specific interests. If you design your own, you must submit a brief, informal proposal for approval.
      • You will not submit the same assessment checklist for the OtS project as you do for the other assessments. Instead, you will submit a project summary including a brief description of the project, a list of the skills you gained/practiced (which do not need to be from the list of learning objectives), and a reflection on the process and outcome of completing the project.
  • Aside from the “menu” provide, you are welcome to design your own assessment! This can be a great chance to cross off some standards that don’t neatly map onto one of the existing assessments or to try out a skill not covered in this class with the “off-the-syllabus” option. Submit your design for approval by emailing me a brief description of the project along with a checklist of which standards you anticipate meeting (or for the OtS option, which off-syllabus skills you are hoping to practice). You must receive my approval before submitting, but I promise I am very, very flexible with this!
  • Typically mini-projects may not be resubmitted, with occasional exception at my discretion. This is typically when I see that just a few very quick and easy fixes are necessary or when the work has significantly missed the mark and should be started from scratch.

Learning objectives

The overall goal of this course is to develop a workflow for producing scientific manuscripts using RStudio and GitHub.

Assessed standards

These 30 standards are assessed in the mini-projects and research project. Each standard is worth 1 point toward your final grade. All 30 must be demonstrated in the final project. At least 20 unique standards must be demonstrated in mini-projects.

GitHub and RStudio

  1. create and maintain a repo with sensible organization and naming conventions
  2. maintain an informative and up-to-date README.md
  3. integrate a GitHub repo with an R studio project, including .gitignore file
  4. effectively use version control
    • creating frequent, informative commit messages
    • relying on document revisions rather than manually created new versions

R programming

  1. find, install, require, and load R packages
  2. use R arithmetic, comparison, and logical operators
  3. parse and define functions and arguments
  4. parse and write conditional statements

Tidyverse

  1. use readr functions to read in and write out data
  2. use dplyr and tidyr functions to transform data
  3. use stringr functions to work with string variables
  4. use forcats functions to work with factor variables

Data visualization with ggplot2

  1. produce 1- and 2-variable plots with geom_* layers
  2. use dynamic aesthetics to group data
  3. use facets to create parallel plots
  4. create publication-quality plots using theme and labs layers

Data analysis

  1. perform simple descriptive analyses for numeric and factor data (e.g., mean, standard deviation, frequency tables)
  2. perform simple hypothesis testing analyses for numeric and factor data (e.g., t-tests, chi-square, linear regression)
  3. present and interpret statistics in manuscript narrative

BibTeX

  1. render APA7 in-text citations with BibTeX syntax with multiple citation forms
  2. render an APA7 references page from a .bib file
  3. dynamically cite R and R packages in-text with cite_r()

Notebooks and code chunks

  1. create and effectively use code chunks following best practices
    • informative names
    • informative comments
    • 1-chunk-1-thing rule
    • distributed throughout the manuscript
    • chunk options
  2. use code chunks to source .R scripts, load packages, set preferences, and read in data
  3. render publication-quality, captioned tables, figures, and images in code chunks
  4. execute descriptive analyses and/or hypothesis testing in code chunks

R Markdown and Quarto

  1. create and maintain an APA7 quarto manuscript YAML header
  2. use Quarto/R Markdown (e.g., headers, text style, lists, etc.) to compose an academic manuscript
  3. use in-text R code to dynamically reference and transform variables
  4. use knitr and quarto to produce an APA7 formatted 1-click PDF manuscript

Unassessed learning objectives

The following learning objectives are not required for the course grade but are important for your development as a data scientist. They may be demonstrated in the mini-projects as part of the additional 20 points earned beyond the 20 for unique, assessed standards.

You are not required to demonstrate these objectives in your research project, but incorporating them can enhance the overall quality of your project and add to the engagement component of the project grade.

Troubleshooting, debugging, and best practices

  1. use frequent, informative comments in code and markdown
  2. find and understand documentation for R packages and functions
  3. follow a debugging workflow with independent and collaborative strategies
  4. parse and investigate error messages
  5. follow a coding style guide

RStudio & R

  1. customize RStudio
  2. use BibTeX with citr and Zotero integration
  3. parse and write for and while loops
  4. create and use intermediate datasets
  5. source .R scripts

Conceptual skills

  1. explain and apply the “grammar of graphics”
  2. name and differentiate R data types
  3. name and differentiate R objects
  4. define “tidy” data and explain its advantages and disadvantages
  5. recognize and interpret common data visualizations
  6. determine most appropriate visualizations and analyses for specific research questions and data