HeroesX: The Ancient Greek Hero: Spring 2013 Course Report (HarvardX Working Paper Series No. 3)

2014
mm/dd/yyyy

Content

Add Comment

What does it mean, to be human? This course takes a close look at the human condition, as viewed through the lens of classical Greek civilization; the basic organizing principle is an objective study of a model of humanity, the Hero.
            —From the HeroesX Course Description

The Ancient Greek Hero is a survey of ancient Greek literature focusing on classical concepts of the hero and how they can inform our understanding of the human condition.

CB22x was offered as a HarvardX open online course in Fall 2013, on edX, a platform for massive open online courses (MOOCs). It was taught by Professor Greg Nagy.

Reich, J., Emmanuel, J., Nesterko, S., Seaton, D., Mullany, T., Waldo, J., Chuang, I., & Ho, A. (2014) HeroesX: The Ancient Greek Hero: Spring 2013 course report (HarvardX Working Paper Series No. 3)
19

Content Type: 

Format: 

Introduction to the HeroesX Project Report

This report describes one of the first courses, HeroesX, offered by HarvardX on the edX platform. The report was prepared by researchers external to the course team, based on examination of the courseware, analyses of the data collected by the edX platform, and interviews and consultations with the course faculty and team members.

The report proceeds in six parts. We begin by describing the goals of the HeroesX course team, in the belief that any learning environment should be evaluated in the context of its intents, values, and vision. We then describe the structure of HeroesX, followed by an examination of the participants who registered for the project. With an understanding of what the course team created and the learners who took an interest in the course, we then turn to examining how participants interacted with the resources, including their patterns of assessment-taking, persistence, and overall activity. We end by examining the limits of our understanding of student learning in these online courses, and the future directions of the HeroesX project.

Our hope is that this report and its companion reports—including a synthetic multiple-course report and other reports from the first HarvardX courses—will inspire new avenues of research and provide insights to future course designers in designing the next generation of open online courses.

The Goals of HeroesX

Since 1978, Greg Nagy has taught a General Education course on concepts of the Hero in ancient Greek literature to undergraduates at Harvard College and to students at the Harvard Extension School. In March of 2013, The Ancient Greek Hero (“HeroesX”) became one of HarvardX’s first offerings, as well as one of the first humanities courses to be offered on the edX platform.

As HeroesX developed, the development team—scholars at the Center for Hellenic Studies and instructional designers at HarvardX—ceased referring to HeroesX as a “course,” and they began calling it a “project.” Repurposing the acronym “MOOC” as a reference to “Massive Online Open Content,” they replaced the term “students” with “participants,” replaced “grades” with “scores” and eschewed other terms commonly used in typical course experiences.

This attention to language is hardly surprising in a project devoted to attention to ‘logos’, the word. Nagy and his colleagues have worked diligently to create an inviting, flexible learning environment that serves the interests of a diverse set of learners, especially those with no background in Hellenic literature. The mission of HeroesX is to invite new learners into a lifetime of study of Greek texts, and to support the learning of those already aboard this journey.

While some edX courses place particular emphasis on advancing through assessments and problem sets, HeroesX deliberately invites participants to learn from the course material however it suits them, whenever it suits them. A message to participants from early in the project reminds participants:

On Monday, 18 March, the first “assessment” exercises will be available to you. If you are experiencing trouble with them, do not feel discouraged. Some people get the hang of reading Greek literature later rather than sooner. It takes practice. In any case, you are free to participate as an "explorer" by simply enjoying some of the content and discussions. That way, you are not obligated to do any of the exercises, and you are simply coming along for the ride. That way of participating is just as legitimate and honorable as the “certificate” option.

To be a certificate seeker, simply complete all the assessments and quizzes in a satisfactory manner by [the final due date]. You are also free to change your mode of participation from “explorer” to “certificate-seeker” **or the other way around**. You can change from one mode to another at any time between now… [and] the last day of the course. 

—March 16, 2013 Course Info Message

One of the central learning goals of HeroesX is to help participants learn to read out of the text rather than reading into the text. To read out of the text is to read Greek literature in its own context, recognizing that the Greeks had, in some cases, very different values than our own. To read into the text is to bring the values of contemporary society into the ancient Greek world and to risk having one’s assumptions cloud the interpretation of words from the past.

This useful reminder applies equally well in the study of large-scale online courses. To evaluate HeroesX properly, observers should make every effort to understand the course on its own terms, especially in its effort not to certify competency in specific technical skills, but to introduce to people to the joys of reading ancient literature and seeing those ancient themes reflected in the works of our own time. 

The Structure of HeroesX

HeroesX is organized around 24 “Hours,” a term that emerged from the 24 hours of classroom time in the residential version of the Ancient Greek Hero. Each Hour in HeroesX includes an overview lecture from Prof. Nagy and a conversation among Nagy and various members of the HeroesX “Board of Readers,” scholars and students who work with Nagy to interpret and comment on the texts. Each week also includes a set of selected passages for “close reading.” These passages are read in performance by a student, and discussed by Nagy and his colleagues. In some Hours, ancient texts are paired with modern media, including film. These comparanda come from sources as diverse as the 1982 film Blade Runner and the 1951 version of the Jacques Offenbach opera Tales of Hoffmann. All of these materials that comprise a single Hour of HeroesX are embedded in a “chapter”—the highest-level organizational unit in the edX courseware.

The video lectures are accompanied by two written texts, both of which were provided to participants for free in digital form. The first is The Ancient Greek Hero in 24 Hours (referred to within the course as “H24H”), a book written by Prof. Nagy and published in 2013 by Harvard University Press. This text, which is based on Nagy’s 35+ years of teaching this specific course at Harvard, provides a combination of reading strategies, overviews of sets of texts, close exposition of particular texts, connections to modern texts, and other insights to aid participants. Each Hour of the courseware has an accompanying chapter (or “Hour”) in H24H. The other text is the Sourcebook of Ancient Greek Texts Translated into English (referred to as the “Sourcebook”). The Sourcebook contains all 55 primary source texts from the course (from Homer’s Iliad to selections of Sapphic verse to Aristotelian philosophy), translated into English by Nagy and a team of scholars. While both books are assigned in their entirety, participants are encouraged to engage the Sourcebook via “fast reading,” pushing through the text without getting hung up on confusing or difficult passages. (From the Advice to Students document provided by the teaching staff at the beginning of the course: “The ancient texts you are reading give you many chances to “get it,” even if you cannot seem to “get it” the first time around.”)

Each Hour of HeroesX also has two sets of quiz questions. The Question Set asks a series of multiple-choice questions that ask participants to define vocabulary terms and recall specific events. The Close Reading Exercises present participants with focus passages from the primary source texts with annotations by Prof. Nagy.  In these quizzes, participants are asked a question of interpretation, such as “Achilles declares that he is ready to die for a just cause. How is this declaration relevant to the possibility of an afterlife for Socrates?” Participants are invited to answer this question in free text, which is not for credit, and then invited to choose the best response out of three fixed responses (essentially a multiple-choice format), which are for credit (two points for “most correct,” or for best “reading out of the text”; one point for “somewhat correct,” or “reading out of the text while also reading into it”; and zero points for “incorrect,” or “reading into the text”).

HeroesX also includes discussion forums. Then intent was that these would be grouped into clusters of 1,000 participants. Unfortunately, in the first attempt at cohorting, edX created three clusters of 1,000 and one of 27,000. Later, in trying to fix this large cohort, they created dozens of much smaller cohorts, with sizes ranging from 12 to 100, fewer than was necessary to facilitate engaging conversation. The decision was then made on May 8, to return all participants into a single forum. 

Table 1. Important Dates in HeroesX

December 19, 2012 Registration Opens
March 12, 2013 Project Start Date
July 24, 2013 All Content Released
August 26, 2013 All Graded Material Due for Certificates
September 8, 2013 Date of Report Data Collection

The project launched on March 12, 2013 and the 24 hours were released as they were completed through July 24. August 26th was the final day that participants could submit assignments and receive credit for certification. For this report, we used registration and tracking log data through September 8, 2013.  

Participants of HeroesX

For HeroesX, edX has records for 43,563 participants who registered for the project. The demographics of participants for HeroesX differs in a few interesting ways from the characteristics of students registered in the other four large-scale HarvardX courses. The gender balance in HeroesX was nearly 50/50, one of the most even ratios among HarvardX and MITX offerings to date. HeroesX participants were older than students in other courses. The median age was 29 and 25% of participants were 40 or older. As with other HarvardX courses, HeroesX participants are highly educated: 36% have a Bachelors degree, an additional 28% have a Masters or other professional degree and an additional 6% have earned a doctorate—combined, over 70% of registrants had at least a four-year college degree. (For more details comparing HarvardX and MITX courses, see the accompanying multiple-course report). These demographics are shown in Figures 1, 2 and 3, where the distributions within HeroesX are compared to the average of the distributions in Computer Science, Justice, Health in Numbers, and Human Health and Global Environmental Change.

Figure 1. Gender distribution of HeroesX registrants (n=39,431; 4,132 missing) and registrants from first four other 2012–2013 HarvardX large-scale courses (n= 309,090; 31,122 missing).

Figure 1

Figure 2. Educational attainment distribution of HeroesX registrants (n=37,908; 5,655 missing) and registrants from other HarvardX large-scale courses (n=330,671; 45,080 missing.

Figure 2

Figure 3. Age distribution of HeroesX registrants (n= 38,975; 4,588 missing) and registrants from other HarvardX large-scale courses (n=342,048; 33,703 missing).

Figure 3

HeroesX participants joined from all over the world. One-third of registrants joined from the United States, and then large contingents also joined from Greece and India. As with other HeroesX courses, large population centers like Brazil and China are also represented with large contingents. The study of the Ancient Greek Hero was truly a global enterprise.

Table 2. Counts and percentages of participants country of residence among 40,119 with identifiable locations. (3,446 missing)

  Count Percent
United States 14,142 35%
Greece 2,874 7%
India 2,365 6%
U.K. 2,003 5%
Brazil 1,564 4%
Canada 1,171 3%
Spain 962 2%
China 824 2%
Australia 690 2%
France 688 2%

Participants enrolled in HeroesX from the day that registration opened and continued doing so even after the submission deadline for the course. Enrollment was brisk in the first two full days of registration, with over 1,000 people enrolling each day. From that point until the course launch day, registration averaged 297 participants per day; after the course launch, registrations averaged 98 participants per day.

Figure 4. Cumulative enrollment for HeroesX (n=43,563).

Figure 4

Researchers of large-scale online courses have devised wide-ranging definitions for interesting sub-groups to study, and in this report we create four subgroups based on two criteria: participant performance on the assessments and participant engagement with “chapters,” the highest level unit of the courseware  (In HeroesX, chapters include the 24 Hours and 3 additional sections of logistical material). Figure 5 shows a scatterplot with participant grade on the Y-axis and the number of chapters opened by participants on the X-axis.

Figure 5. Scatterplot of grades versus chapters viewed for HeroesX registrants (n=43,563).

Figure 5

Notice that in Figure 5, nearly all points fall below the diagonal. In the courseware, the quiz questions are nested within each Hour or chapter, and the quiz questions are roughly equally weighted and comprise the entirety of the grade.  Therefore, because there is a nearly perfect linear relationship between the maximum number of points one can achieve and the number of chapters that are accessed, almost all points must fall below the diagonal.

Within this scatterplot, we identify four groups of interest. Above the cutoff score of .5, participants earned a certificate. Among certificate earners, we spotlight two special cases. The “completionists” are those few participants who earned a perfect score on all assignments and in doing so viewed all of the chapters. The “optimizer,” at the lowest left point above the certificate cut-off line on the diagonal, was the account that earned a certificate with the lowest possible passing grade while opening only 12 of the 24 Hours of the course.

Among those who did not earn a certificate, those in the bottom right quadrant of the scatterplot opened and viewed at least half of the chapters of the course. In the bottom right, we highlight the “listener,” the participants who answered zero assessment questions correctly and viewed the entire content. In the bottom left quadrant are participants who viewed less than half the chapters in the course. In the very bottom left are participants who never viewed any course chapters.

Figure 6. HeroesX participants clustered into four disjoint sets (not to scale). (n=43,563).

Figure 6

From this variation, we define four sub-groups of HeroesX participants. Registrants are the 43,563 participants who registered for the course. Of these 25,686 viewed the courseware section of the course site where the course videos, assessments, and other core content are located. Of these, 1,773 viewed more than half the chapters in the course; we call these participants explorers. Finally, 1,400 participants are certificate earners who earned a grade of 50% or higher. In Figure 6, we present these groups as disjoint sets, noting the numbers of participants who only viewed (neither exploring nor certified) and only explored (without certification).

Those who persist furthest in the course differ demographically from the registrants and viewers. The median age of all registrants was 29, while the median age of explorers and certificate earners was 36.  Figure 7 shows that participants who explore more than half the course or earn a certificate are more highly educated and more likely to be female.  While women made up 47% of all registrants, they make up 51% of certificate earners. Among all registrants, 70% have a Bachelor’s degree, compared to 77% of certificate earners.

Figure 7. Distribution of females and participants with Bachelor’s degree among all registrants and in four disjoint groups of HeroesX participants (n=43,563).

Figure 7

Participation and Activity in HeroesX

In examining the participation data from HeroesX, it is important to remember our early admonition to read out of rather than into the project. HeroesX offered a long and intellectually demanding experience, and participants were invited to participate as they saw fit. The HeroesX team welcomed the casually interested to learn alongside the committed Hellenic scholar. Any evidence of learning activity is regarded as advancing the goals of the project. While public discourse around massive open online courses has often focused on narrow measures of completion and attrition, it is perhaps more important to understand all of the ways that learners can participate and contribute to these learning environments. 

Grades and Certification

Participants who earned a grade of 50% and above were awarded a certificate of completion. Out of all registered participants, 4,681 answered at least one problem correctly and earned a grade. Most who were assigned a grade only dabbled in the assessments of the course; the median grade (not including those with a grade of 0) was 9%. A total of 1,400 participants earned at least a grade of 50% and were awarded a certificate.

Figure 8. Distribution of grades across all HeroesX participants who answer at least one for-credit question correctly (n= 4,681).

Figure 8

There are numerous ways to calculate a passing rate for a large-scale online course. None of these can be interpreted like a dropout rate or failure rate of a residential course, given the vast diversity of project participants and the different goals of the instructors. We do not possess data that would allow us to identify the participants in our sample who have the same level of commitment and preparedness to learn as students in a residential course. However, as a set of purely descriptive statistics, we can take the number of participants who earned a certificate (1400) and divide that number by the following:

  • the number of participants who registered (43,563), yielding 3.2%;
  • the number of participants who registered before the final due date (43,418), yielding 3.2%;
  • the number of participants who viewed the course and registered before the final due date (25,611), yielding for 5.5%;
  • the number of participants who viewed more than half the course (1,778), yielding 78.7%.

The registration for HeroesX opened 80 days before the course launched, and the course lasted over five months, from March through the end of August. We took an interest into whether students who registered particularly early or late were less likely to earn a certificate in the course than those who register near the official course launch. In HeroesX, we do find a modest relationship between registration date and passing rate. Participants who registered long before or long after course launch were slightly less likely to pass the course than those who registered near the launch.

Figure 9. Average trend line (loess approximation) of pass rate (certified participants over all registered) by daily registration cohort (n=43,563).

Figure 9

Figure 9 shows an average trend line (loess approximation) of passing rate by daily registration cohort. For the early registrants, one hypothesis is that a larger number of these registrants lose interest in the course in the long waiting period. For the later registrants, the surprisingly similar passing rates suggest that those who are motivated to become certified can achieve their goals in a short amount of time, though it becomes more difficult as time grows shorter. These are only hypotheses, however, and the field of open online learning has much to learn from explorations of those who register for courses at times that would not be possible in a typical residential or online course setting.   

Activity in HeroesX

We can gain greater insight into how participants experienced HeroesX by looking at their levels of activity. In this section we examine four measures of participant activity, the chapters they viewed, the problems they viewed and attempted, their video watching patterns, and their total number of actions.

A simple summary of participant activity with a course on the edX platform is the number of chapters opened by a given participant, shown in Figure10. Chapters are the highest level unit in the courseware. HeroesX had 27 chapters, one for each of the 24 Hours and then 3 logistical chapters (“Coming Soon,” “Hour 0: Introduction,” and “Afterward.”) The number of chapters viewed by a participant offers a baseline insight on what portion of the courseware they could have engaged with. (Together, Figures 8 and 10 represent the marginal distributions of the variables comprising Figure 5.) The distribution of chapters viewed for HeroesX shows a bi-modal distribution typical of many activity distributions of course on the edX platform. Most who view the courseware view only a handful of chapters. Very few view a middling number of chapters, and then we see a secondary mode around the ceiling of 27 chapters, representing those who at least opened all of the chapters in the course.

Figure 10. Distribution of chapters viewed for HeroesX participants who viewed courseware (n=25,686).

Figure 10

We can further probe activity by looking at behaviors within those chapters. Figure 11 shows the distribution of the views and attempts at the problems within each Hour of the course. Here, we see a sharp drop in activity after week two. After that declines in problems attempted is modest; about 2,000 people attempt problems in Hour 5, about 1,500 in Hour 13, and about 1,300 in Hour 24. The number of people viewing problems falls more steadily, from about 6,526 in Hour 5 to 4,462 in Hour 13, to 2,564 in Hour 24.

Figure 11. Average number of participants viewing and attempting for-credit problems in each Hour.

Figure 11

One of the of the central advantages of contemporary large-scale online courses relative to previous generations of distance learning experiences is the availability of streaming video, and HeroesX includes over 250 clips of readings, exposition, and discussion. In Figure 12 we show the log distribution of video play events (the total number of times a participant pressed play in the edX video player) for certificate earners and participants who did not earn a certificate. This does not capture the full range of video viewing behavior, since students can also download videos from the site, but it does provide an introductory view of how the courseware was used.

Figure 12. Video play events as captured by edX video player log data for 1,307 certificate earners and 15,008 non certificate-earners who viewed at least one video in the edX player.

Figure 12

As to be expected, certificate earners watch videos more often than those who did not earn a certificate. The high numbers of video plays in the right end of the distribution suggests that the most engaged participants were watching material multiple times or pausing and restarting often. There were 184 participants in HeroesX who played videos more than 1,000 times. There were also a substantial number of certificate earners who watched very few videos, and 93 certificate earners who have exactly 0 recorded video play events. Some of these participants might have downloaded the videos, while still others might have passed the course by using only the written texts rather than the video components. Even this cursory examination of video usages suggests the very diverse ways that HeroesX participants took advantage of the opportunities available in the course.

One of the oft-mentioned advantages of online courses is the ability to “pause” videos to reflect on a point or repeat a section of lecture that is unclear or difficult. One simple way to evaluate how often this feature is used is to calculate the ratio of video play events to video pause events. Of the 18,168 participants who both played a video and paused a video, 90% had a play to pause ratio of 4 or lower. That is, 90% of HeroesX participants who watched a video clicked pause at least once for every four times they clicked play.

We can get a related perspective on activity by looking at the number of clicks or actions that each participant in HeroesX takes. In Figure 13, we see that most registrants on the site take fewer than 100 actions, but a small number of the most engaged participants click within the course site tens of thousands of time. Similar to Figure 5, this shows that there are some certificate earners who can pass the course with only a few hundred actions and listeners who do not pass the course but take thousands of actions. In general, however, those who do many actions in the course are more likely to earn a certificate than those who do little, a simplistic insight that echoes findings from other early studies of lecture-based online courses. 

Figure 13. Number of participant clicks (i.e. recorded actions) plotted on a log scale for Heroes certificate earners (n=1,400) and non-certificate earners (n=42,163).

Figure 13

Persistence

In addition to measures of learner actions, we can also examine participation through the lens of time. Several factors make persistence complex to calculate and articulate in HeroesX. First, participants can enter the course at any point.  Second, there are no deadlines, so late registrants can complete all of the work of the course, even those who register on the final day of the course. The official launch of the course was March 12, 2013; course content was released every week or two through July 24, and the all responses to questions for credit were due on August 24. Some participants took the course on the pace that the content was released, completing two lecture units every week from March through May. Participants could also sign up for Heroes in August, and, to borrow a phrase from contemporary television viewing habits, “binge-watch” the videos and complete all assignments over a weekend.

As a result, two scales of time are of interest. Absolute time represents time from the launch of the course until its completion.  Week 0 represents the official launch of the course; week -12 represents the first week where participants can register for HeroesX; the exam was launched in week 19; and all graded materials were due in week 23. Relative time is calculated from a participant’s enrollment, beginning with course launch or their registration, whichever is later. For a participant who registers in the week of course launch or before, relative week 0 is the same as absolute week 0, the launch of the course. For a participant who registers in absolute week 2, week 2 becomes their relative week 0. Absolute weeks allow us to ask, for example, whether many participants drop out after the second quiz.  Relative weeks allow us to ask whether participants are most likely to drop out in their first or second week in the course.

To assess persistence, we first examined participants by their registration cohort:  clustered by the absolute week in which they registered to join the course. We then, for each cohort, created a “hazard function” describing the risk of stopping out of the course with relative time on the x-axis. The “hazard proportion” at each time period takes all the participants who have been active up to that time period and asks, of these participants, what percentage will post their last activity this week, without earning a certificate?  In the Figure 14, we plot these hazard functions by registration cohort on relative time. We show every other cohort, from those who registered 12 weeks before course launch (darkest) to those who registered 18 weeks after course launch (lightest), truncated at week 23, when all course materials were due. Cohorts that begin after the course launched (week 2–week 18) have dashed lines.

Figure 14. Hazard functions for every other registration cohort (from absolute week -­11 to 19, darkest to lightest) plotted on relative course week, where week 0 is the week of course launch or the week of initial registration, whichever came later. Dashed lines are for cohorts after week 0. The hazard functions are truncated at absolute week 23, when the all graded material was due. Hazard probability represents the proportion of all participants who ceased activity without earning a certificate of the participants remaining in each time period.

Figure 14

Examining these hazard functions, they are remarkably similar. Of the cohorts that registered at or before the course launch (solid lines), where relative time is the same as absolute time, the hazard functions are very similar. There appear to be certain weeks that have higher hazard than others, and the risk of dropping out applies evenly across all cohorts. These weeks should be of great interest to the designers of HeroesX, and the phenomena of interest to all instructional designers. Notice also that the hazard functions of the cohorts registering after launch track the other cohorts over the first few weeks—the phenomena of high hazard in the first few weeks followed by lower levels of hazard applies regardless of registration time.

For clarity on the fluctuation in hazard from absolute week to week, we also plot these hazard functions on absolute time in Figure 15. Here the hazard functions converge in the later weeks and vary earlier on (we refer to the steep lines on the left side of this figure as the “rake;” the rake is on the left when plotted on absolute time and on the right when plotted on relative time.) This figure more clearly shows how hazard fluctuates over the course. HeroesX is particularly interesting to examine in this regard because the overall structure of the course does not vary from week to week. Every week there are videos, readings, close reading exercises, and questions, albeit in varying volume.

Figure 15. Hazard functions for every other registration cohort (from absolute week -­11 to 19, darkest to lightest) plotted on absolute course week, where week 0 is the week of course launch. The hazard functions are truncated at week 23, when all graded material was due.

Figure 15

Changing hazard as a function of course structure—common in courses with exams and due dates—can be ruled out here. Changes in hazard from week to week must be from changes in the quality of course content, the calendar (holidays, etc.), issues with the edX platform that appear from time to time, or other variables. In HeroesX, we see spikes in hazard in weeks 5, 8, and 12. This is only speculative at this point, but one explanatory hypothesis is that these weeks correlate with the releases of some of the Hours that had substantially more content (in terms of numbers and hours of video length) than other Hours in the course. More detailed investigation is required, but this one detail suggests how examinations of these kinds of hazard models could help course teams tune and refine their syllabi from one course run to the next.

To summarize these hazard functions and display a survival curve, in Figure 16 we plot the average hazard function over relative course week. From this average function, we then directly calculate an “implied survival function.” As displayed here, the survival function drops sharply in the first week, and then the cohort size decreases more slowly from week to week. One way to summarize the figure is to say that regardless of when a participant registered, hazard was highest in the first two weeks, then fluctuated between 5% and 15% of the cohort dropping out in any given week. Put another way, once participants persisted past the second week, their risk of dropping out decreased considerably.

Figure 16. Hazard function comprised of the average of all registration cohort hazard functions plotted on relative course week (where week 0 is the course launch or initial registration week, whichever is later) and truncated at week 23 when all graded material was due. Implied survival function is calculated directly from the average hazard function.

Figure 16

These hazard models give some sense of how participants persist throughout the whole course, examining the span of time from first action to last action. However, a participant can log in only twice, on the first day and the last day, and be counted as “surviving” through almost the length of the course. This motivates an alternative metric that counts how many discrete days participants view course material. Among all registrants, we find that median number of days of activity is one, and 75% of registrants have four or fewer days of activity.

For a more granular view, therefore, in Figure 17 we examine the daily activity of those who have viewed over half the course (those who “explored”) and certificate earners. In the figure, we see that people make substantial investigations into the course over different time periods, ranging from only a few days to nearly 200 days with activity. There are five participants who earned a certificate in a single day of activity, and one certificate earner with activity on 198 discrete days. Similarly, many participants viewed content from over half of the chapters of the courseware in a single day, and several others engaged with the course on over 150 discrete days without earning a certificate.

Figure 17. Days with activity, the number of discrete days (demarked in UTC time) during the observational period where participants had at least one action, for explorers and certificate earners (n=1,733)

Figure 17

Learning in HeroesX

Perhaps the most important question to ask of any course is: what did participants learn? In answer to this question, we can only offer cautions about our present understanding and ideas for probing this question in the future. The note of caution is this: the data we have presented here about activity, persistence, grades, and certification gives us some indication of the quantity of participant’s activity in the course, but little about its quality. We can characterize in great detail the exposure that participants had to learning opportunities; we can say little about what participants made of those opportunities.

This report represents only an initial analysis of the HeroesX data, and with data better cleaned and organized, we can expect researchers to profitably investigate them further. In particular, analyses of the close reading exercises and the discussion forums might reveal how participant thinking evolved during their experience. Still, these data have their limits. The close reading and question sets are multiple choice questions with three possible answers. The discussion forums are rich and generative, but even among those who explored half the courseware, only half contributed to the forums even once, and 75% had fewer than 10 forum contributions or updates. That said, 99 people had over 50 forum contributions or updates, and they would be an interesting group to study, even if they are not representative of the whole.

Devising new mechanisms to better understand what HarvardX participants are thinking and learning is vital to fulfilling the research aims of the enterprise. Beginning in the fall of 2013, the research committee launched a HarvardX-wide survey infrastructure to support pre-course and end-of-course surveys, to get a better sense of participants’ incoming proficiency and learning. Additional innovations in online assessment will be essential to understanding learning at scale. For instance, the HeroesX team is working with HarvardX in supporting the development of annotation tools that could provide insight into how learners are making meaning of texts as they read them.

The Future of HeroesX

Just eight days after the first iteration of HeroesX wrapped, the second one began. The Fall 2013 version of the HeroesX project launched in September 2013 to support three groups of learners. In addition to participants who signed up for HeroesX on edX, the courseware is also being used by students from Harvard College enrolled in the General Education course on campus, and students both in Cambridge and online who are enrolled in the course through the Harvard Extension School. All three groups of learners have embarked together on another experiment in learning, sharing the same course materials and discussion forums.

One new innovation to bring all three groups together is the Discussion Circle. In the residential setting facilitated by Professor Nagy, a group of students are invited each week to hold a “fishbowl discussion” in front of the rest of the class, a mechanism to invite a smaller group of students from a large class into a more focused dialogue. These discussants are then required to act as provocateurs in a forum discussion thread, where learners from beyond the college are invited to engage in a dialogue on the issues of the week. This is one example of the ways that HarvardX can connect students in Cambridge to learners in the wider world.

In the Afterword to the first version of HeroesX, Professor Nagy ended with these words:

In terms of [telos], I should truly say that this course has not come to an end, we simply have come full circle. And one of the things I do hope that many of you will do, maybe all of you, is—don't laugh—take the course again. And I don't want to call it a course, up to now I haven't. Shall I say it a different way? Take the project again. Experience it again. Read all the things that you've read again. Think about them again.

But if you don't have a chance to do that, I still say you should make yourself open for further encounters with this beautiful set of knowledge and experience…. So is this the end? I don't think so. And I do hope that one way or another we can maintain contact with each other. And that whatever experiences you have will be affected, even connected, to the world of heroes that you have experienced so well.

Not surprisingly, not every participant from HeroesX re-­registered, but as of November 29, 2013, there were 4,326 people who did, including 419 who had earned a certificate in the first HeroesX and 146 participants who had explored more than half the content without earning a certificate. Some of the most engaged participants were invited to serve as Community Teaching Assistants in the discussion forums to help others on their learning journeys through these resources.

For centuries, the texts of the ancient world have promised rich rewards to those willing to commit to a lifetime of study. Through HeroesX, Professor Nagy and his colleagues on the course team have extended that promise, and many have taken them up on the offer.