Athletic Software Engineering (2017)

17 Jun 2017

For the past four years, I have been developing a pedagogy called “Athletic Software Engineering” (ASE) which I use in ICS 314: Introduction to Software Engineering. This pedagogy involves a high intensity, time-constrained, and often stress-inducing approach to acquiring competency with software engineering skills. Despite those characteristics, a survey of ICS 314 students from AY 2016-2017 found that 90% of them prefer athletic software engineering to a more traditional teaching approach.

This essay presents an overview of Athletic Software Engineering as I currently teach it. I begin by explaining what motivated me to design the pedagogy. I follow this with a summary of the major features of Athletic Software Engineering, and conclude with the results of student evaluations of the method and my class.

1. Motivation

To motivate athletic software engineering, it’s necessary to first understand the goals of my software engineering course:

  1. Acquire competency with a minimal technology stack for modern software application development. In the case of ICS 314, this “minimal” stack includes Javascript, git, GitHub, ESLint, IntelliJ, HTML, CSS, Semantic UI, markdown, mocha, chai, npm, Meteor, MongoDB, and Galaxy.

  2. Experience “design” problems. Design problems occur at the code level (for example, how to structure code so that it is testable), as well as at the user interface level (for example, how to organize the presentation and manipulation of data so that user needs are satisfied).

  3. Experience “process” problems. Process problems involve group communication and coordination, and how to define, delegate, and manage tasks such that a group of developers can work together to create a functional application in a limited amount of time.

  4. Design and implement an application that solves a “real world” problem for UH students. For example, students on a meal plan are allocated a certain number of “points” each week for buying food, which disappear if unused. To solve this inefficiency, one team developed an app called “Manoa Dining Delivery”, in which students can turn their expiring points into cash by purchasing food at the cafeteria and delivering it to another student on campus. Here is a portion of their home page:

These four goals for ICS 314 build on each other: without competency with a technology stack, it is difficult to experience significant coding and user interface design problems. Significant design problems are needed in order to provide a context for teams to experience process problems. And without the ability to solve both design and process problems, it is difficult to successfully design and implement a “real world” application.

Note that developing an application to solve a “real world problem” requires a substantial portion of the semester. I find students need a minimum of six weeks to work on the final project, which leaves only 10 weeks to develop “competency” with the underlying tools and techniques. Athletic Software Engineering is designed to solve the pedagogical problem of developing competency with a significant set of tools and technologies in the time available during a semester.

2. Why “competency” is important

Consider the following development scenario, which is typical during final project development:

A student chooses a development task (Issue 23) from the GitHub Project board, and moves it from “Backlog” to “In Progress”. She then creates and publishes a new git branch called “Issue-23” off the project’s master branch in which to accomplish the task. The task involves the creation of a new page containing a form in the application. She creates a new page template in Meteor along with a route to that page, several Javascript callback functions to manage form processing, and a responsive layout for the page using HTML, CSS, and Semantic UI. To store the results from the form, she extends a MongoDB collection with new fields, and writes a test using mocha and chai. In the IntelliJ IDEA editor, she notes any ESLint code style errors as they occur and removes them. When the page works correctly, she commits her branch to GitHub, merges the results into the master branch, and moves the task from “In Progress” to “Done”.

Notice that in this (very simple) development scenario, there are nevertheless explicit references to a dozen technologies (GitHub, git, Meteor, Javascript callbacks, HTML, CSS, Semantic UI, MongoDB, mocha, chai, IntelliJ, and ESLint). There are also implicit references to design problems (What information is being manipulated on that page? Is that the right information to manipulate? Is the chosen page the appropriate place to manipulate that information?) and process problems (Is that the appropriate task for that developer to work on right now? How might that task interact with others being worked on by others in the group? What communication should be occurring to make sure the developers do not implement conflicting code?)

One key insight motivating athletic software engineering is this: if an ICS 314 student is competent with the technology stack, then and only then is it possible for that student to actively engage with “design” and “process” problems. If a student is not competent with the technology stack, then their focus devolves to understanding the technology stack, not the higher level “design” or “process” issues. For example, if a student does not understand how to implement a form in Semantic UI, then that becomes the focus of attention; it is unlikely that the student will spend the time to compare and contrast design alternatives without an understanding of how to implement any of them.

3. The intrinsic complexity of a software engineering technology stack

One question is whether ICS 314 requires an overly complex technology stack. I strive to require as minimal a technology stack as possible, such that whatever complexity results is intrinsic to the practice of software engineering. Consider that the 12 technologies can be organized into the following seven categories, which represent central concepts in software engineering:

  1. Programming Language (Javascript)
  2. Development Environment (Intellij IDEA)
  3. Project Management (GitHub issues and project boards)
  4. Configuration Management (git branch and merge)
  5. User Interface (HTML, CSS, Semantic UI)
  6. Testing and quality assurance (mocha, chai, ESLint)
  7. Application framework (Meteor, MongoDB)

Over the years, I have taught ICS 314 with many different combinations of technologies (Java rather than Javascript, Play rather than Meteor, Bootstrap rather than Semantic UI, Subversion rather than git, Google Project Hosting rather than GitHub, JUnit rather than mocha/chai), but these seven classes of software engineering technologies persist, as well as the need to obtain competency with them in order to experience “design” and “process” problems.

4. Efficient and effective development of competency

Athletic software engineering is a term I created to describe a set of interrelated teaching practices whose goal is to help students develop “competency” with basic software engineering skills prior to commencing their final project. If they achieve competency with most or all of the skills, then the final project: (a) enables them to develop experience solving design and process problems; (b) makes it more likely that they will successfully implement a “minimum viable product”, and (c) results in a significantly richer educational outcome.

While a description of athletic software engineering sufficient for classroom implementation is beyond the scope of this essay, here are some key features that distinguish it from other approaches.

Weekly module schedule. The class is organized into a sequence of “modules”, each lasting one week. If my class meets on Tuesdays and Thursdays, then each module begins on Friday and lasts until the end of class on the following Thursday. Students work on “practice WODs” from Friday to Monday, do a “group WOD” on Tuesday, an (optional) Amy WOD after that, and an “in class” WOD on Thursday to complete the module. (These WODs are described in more detail below.) Here’s an illustration:

Javascript 2 is an example of a module.

Skill acquisition, not concept acquisition. Students are evaluated based upon their ability to complete a task, not on their ability to discuss a concept. For example, a test module would evaluate the ability of a student to define and implement a unit test, not evaluate the ability of the student to describe the difference between a black box test and a white box test.

Workout of the day (WOD). I use the CrossFit term “WOD” in Athletic Software Engineering. In ASE, a “WOD” is a problem that requires the student to exercise one or more skills both correctly (i.e. such that the problem is solved) and efficiently (i.e. within a time limit). There is an upper bound on the time allowed to solve a problem (which typically makes it impossible to “google your way to the answer”.) To set the upper bound, I time myself solving the problem to establish an “expert” time, then add 50% to 100% more time to obtain the upper limit for students. This upper bound is referred to as the “DNF” time (or “Did Not Finish”). This past year, I assigned around 35 WODs each semester, which ranged from 5 minutes to 40 minutes to complete, with most WODs having a DNF time of somewhere between 10 and 20 minutes.

Practice WODs. Each module provides several “practice WODs” to be done as homework between Friday and Tuesday. Practice WODs differ from traditional homework assignments in several ways. First, each practice WOD indicates, in addition to the problem to be solved, the time limit for successful completion of the WOD. Second, each practice WOD includes a video of me solving the problem, and timing myself while doing so. Third, I typically instruct students to first attempt to solve the problem by themselves without watching my solution video, and time themselves while they do so. If they have not solved the problem by the DNF time, then they should stop at that point and watch my solution video, then try to solve the problem again from scratch, timing their second attempt as well. If they cannot complete the assignment before the DNF time again, they are instructed to review the video and keep attempting to solve the problem until they can do it within the specified time limit.

Some students admit that they do not follow this procedure, although most learn quickly that this is actually the most efficient way to acquire competency and avoid the dreaded DNF on the “in class WOD” (see below).

Jamba Juice 1 is an example of a practice WOD.

Group WOD. The Tuesday class is dedicated to one or more “Group WODs”. For each Group WOD, I randomly split the class into groups of two. I then provide a new problem for them to solve using the same skills they (hopefully) developed from the practice WODs. However, they are now working in a group, and they have to work concurrently (in most cases, I ask that they both complete typing each line of code before either can type the next one.) Once again, there is a time limit, although the limit is elevated since working concurrently can slow things down, and because I want to encourage them to discuss issues as they arise during the solution process.

The Group WOD fulfills several pedagogical goals. First, it provides a self-assessment procedure for students; they can see if their work on the practice WOD has actually prepared them to successfully solve a new problem. Second, it provides an engaging classroom learning environment; rather than working silently alone, they get to work with another student. By the end of the semester, most students have had 1-on-1 time with most other students in the class. Third, it incentivizes students to do the practice WODs, since they know that they will need to demonstrate the skill in class in Tuesday with another student, and this creates social pressure to prepare. Finally, doing the Group WOD helps the students acquire more competency with the skill itself.

BK Menu is an example of a Group WOD.

Amy WOD. During academic year 2016-17, my TA’s first name was Amy and she held a weekly, voluntary meeting to provide another opportunity to do a timed WOD. Amy WODs are individual and timed, and after the conclusion she goes over the solution. Amy WODs were usually on Tuesday or Wednesday afternoons or evenings, after the students had (hopefully) finished the Practice and Group WODs.

Pokemon is an example of an Amy WOD.

In class WOD. The Thursday class period ends the week-long module, and it consists primarily of the “in class WOD”, which is yet another individual WOD, but this time it “counts”: almost all of the points for the module are based upon the student’s ability to solve the problem presented in the In Class WOD both correctly and within the specified time limit. In class WODs are generally graded on an “all or nothing” basis: if you solve the problem correctly and within the time limit, you get 100% of the available points. If you don’t, you get zero. Early in the semester, it is possible for up to half the class to DNF on the in class WOD. As students learn how to acquire competency effectively, this percentage tends to drop. Most semesters there is at least one in class WOD where no student DNFs.

Bar Tab is an example of an in-class WOD.

The combination of extensive “training” throughout the week in the form of practice, group, and Amy WODs, followed by a single “performance” to demonstrate the skill, is similar to many athletic (and artistic) disciplines. It is designed to help students learn to take the “training” seriously, just as athletes (and artists) take training seriously, because it is the only way to increase the odds that they will succeed at “game time”.

5. Student evaluation of Athletic Software Engineering

Most students report that the WOD-based athletic software engineering pedagogy is very stressful. To obtain a detailed perspective, I ask students each semester to complete a questionnaire about their view of WODs and Athletic Software Engineering. Here are the results for AY 2016-2017.

There were a total of 119 students across the four sections of ICS 314, and 86 filled out the survey, for a 72% response rate. The following sections present data from the most relevant questions:

Do you prefer ASE to the traditional way of teaching?

90% of the students surveyed prefer Athletic Software Engineering. Positive comments include:

About one out of ten students did not prefer ASE. Here is a comment from one who didn’t:

Are repeating the WODs helpful?

84% of the students surveyed indicated that repeating WODs were helpful either all the time or part of the time. Positive comments include:

Here’s a comment from a student who did not find repeating the WODs to be useful:

Was the in-class, graded WOD helpful for learning?

91% of the students surveyed found the in-class WOD to help in learning. Positive comments include:

Here is a comment from a student who did not find the in class WOD helpful:

Did WODs help you become more confident in your software development capabilities?

80% of the students surveyed responded that WODs increased their confidence. Comments include:

Those who did not find it helpful noted:

Did WOD-based homework help you to be more focused?

79% of the students surveyed responded that WOD-based homework helped them to be more focused. Comments include:

On the other hand:

6. Student evaluation of ICS 314, AY 2016-2017

As noted above, the “athletic” portion of the class covered approximately the first 10 out of the 16 weeks of the semester. The final six weeks focused on applying these skills to the development of a real world application, and (hopefully) wrestling with the resulting design and process problems. So, some additional insight into ASE might be gained by reviewing the end-of-semester course evaluation data. Since 2007, I have been publishing all my course evaluations without any editing, and they are available for review here. The following presents a subset of AY 2016-2017 course evaluation data that seems most relevant to this essay.

119 students completed ICS 314 across four sections during AY 2016-2017. 115 of them completed the end-of-semester course evaluation, for a response rate of 97%.

There are three Likert Scale questions that seem particularly relevant to this essay:

Statement Strongly Disagree Disagree Neutral Agree Strongly Agree
The instructor fulfilled the goals of the course 1 1 1 30 81
I gained a good understanding of concepts/principles in this field. 1 1 9 29 75
The instructional materials (e.g., texts, handouts, etc.) were relevant to course objectives. 1 0 3 32 77

These responses indicate that at least 90% of the students surveyed agreed or strongly agreed with each of these statements.

The free response data is harder to summarize in a useful manner. Here are two comments that I find particularly revealing:

7. Conclusions

After four years of using Athletic Software Engineering in the classroom, I find it to be the most effective method I have discovered so far for teaching students the basic skills necessary to experience real world software engineering design and process problems. Perhaps counter-intuitively, student feedback is very positive, despite the pressure and intensity of the pedagogy. This, ultimately, does not surprise me: as long as the rewards appear commensurate with the struggle, most students are willing to work hard.

That said, I see many, many opportunities for improvement to the course. I want to develop technology to simplify the development of professional portfolios, which should resolve some of the problems encountered by some students. I want to provide more practice WODs, and improve the pedagogical quality of the current WODs. There are some additional software engineering concepts (including continuous integration, performance evaluation, and design patterns) that I would like to introduce if time permits. Just keeping the materials current with the fast moving world of software engineering and application development requires constant investment of time and energy.