Physics Honors: MasteringPhysics

From an e-mail I sent my student & parents:

As you know, this year in Physics Honors we are using an online physics homework solution called MasteringPhysics. Students work online tutorials and practice problems that are automatically graded. It is worth looking into whether such a tool is effective.

The website for the program is www.masteringphysics.com -- parents, feel free to ask your son or daughter to show you around!

Following is a quick analysis I did of student performance on quizzes & exams in Physics Honors & how that performance correlated with hours logged & total score on MasteringPhysics (MP).

The five plots in the .pdf file attached (also available on my website) are as follows:

UPPER LEFT: Average Quiz Score vs. Hours Logged on MP
UPPER RIGHT: Average Quiz Score vs. Fraction of Total Points Scored on MP

LOWER LEFT: Average Exam Score vs. Hours Logged on MP
LOWER RIGHT: Average Exam Score vs. Fraction of Total Points Scored on MP

BOTTOM: Fraction of Total Points Scored on MP vs. Hours Logged on MP

(Remember that you can zoom in on this .pdf file to see it better.)

A few things stand out immediately:

- Hours logged in MP seems to have no bearing on quiz and exam scores. This is not surprising: some folks multitask while logged into MP, some folks are working in a very focused way. Some people find the answers come quickly to them and do well, others spend very little time in MP but also do not understand the material well. In fact, you can see on the bottom-most chart that students who logged as few as 6 total hours over the semester scored about the same on MP as those who logged 17 hours. There does seem to be a cutoff -- students who did fewer than 6 hours of work in MP did poorly on quizzes & exams. This might be more reflective of overall engagement in the course.

- No student earned more than 90% of the total points on MP -- this is also not surprising, given that we awarded grade credit based on completion: typically if a student earned about 60% or more of the points, and put in non-trivial amounts of time on it, they earned credit for the assignment.

- Total points scored on MP seems to correlate STRONGLY with quiz and exam scores. Nearly every student who earned above about 75% of the points on MP earned average quiz grades of B- (2.5) or higher. Nearly every students who earned below about 50% of the points on MP earned average quiz grades below B-. A similar strong correlation is seen in the exam grades.

As you know, just because two things are *correlated* (meaning when one increases, so does the other) doesn't mean that one *caused* the other. More on this idea here.

However, since the same people (Mr. O'Keefe and myself) chose the MP software, chose the MP problems, and wrote all the quiz & exam problems -- and we did all of this to get at the same underlying basic ideas -- I think these data are strongly suggestive of MP's overall positive impact on quiz & exam scores.