Author Archives: David Mullich

Tracking Down Game Bugs

As a development team nears completion of a game’s features and assets (art, audio and text), the game starts to take its final form, but it is likely still rife with errors and problems that we call “bugs”. The term “bug” for a software error came about in 1946 when engineers traced an error in a Mark II computer to a moth trapped in a relay. This bug was carefully removed and taped to the log book. Stemming from the first bug, today we call errors or glitches in a program a bug.

Development teams send off versions of their game to a fresh set of eyes for finding the bugs. These set of eyes belong to the Quality Assurance team, also called “testers”. Testers use check sheets to go through the game and ensure that all the thousands (even millions) of features and assets that were supposed to be implemented are actually there, working as planned.

When a Quality Assurance team reviews a game build and finds bugs, they write up a report for each bug found. Bug reports are generally written in an online database with the following information:

  • žBug #: This is automatically generated for each new report. It is much easier to refer to Bug #725 than “you know that problem in the Valhalla level where in the green building in the back where two walls don’t quite fit together properly?”.
  • Summary (Headline): A synopsis of the problem.
  • žLocation or Component: Where in the game the problem occurred.
  • ž

  • Description: More detailed information about the bug.
  • žExpected Result / Actual Result: For when bug is not obvious (for example: after reloading, the gun should have been able to fire 8 shots, but the actual result was that it could only fire 6 shots).
  • Steps to reproduce: So that the development team can replicate the bug. Unless a team can recreate a problem and see it for themselves, they may not be able to fix it.
  • žReproduction Rate: How frequently the bug occurs. Does it occur all the time (100%)? Have the time (50%)? Is it not repeatable.
  • ž

  • Severity: Bug severity levels are usually categorized as:
    • A = (Blocker / Critical) Fatal flaw. —Crashes, freezes, can’t finish game.
    • B = (Major / Normal) Serious flaw. Features don’t work properly.
    • C = (Minor / Trivial) Minor flaw. —Glitches in artwork, typos, minor annoyances.
  • žPriority: Does the problem need to be fixed ASAP? Can it be saved for a patch? (Priority is usually determined by the project’s producer and not the QA team).

This information is sent back to the development team. If they address the issue, they will mark it as FIXED and the QA team will retest and verify the fix when they receive the next build.

The team may also mark a bug as CAN’T REPRODUCE or NEED MORE INFORMATION if they can’t duplicate the problem. Or, if they think the game is working as designed and the QA team just disagreed with the design direction, they may mark the bug as NOT A BUG or WORKS AS DESIGNED. Finally, if the producer decides that this is a minor problem that can ship with the game, it may be marked as WILL NOT FIX.

The bug database is preserved after the product ships so that it can be referred to when making patches and updates to the game.

It is not unusual for thousands of bugs to be found during the course of game testing. And even when using a bug database to track down every single error found, some do slip through the cracks and wind up bugging the customer.

 

 

Our Favorite Hobbit And Gamer, Sean Astin, Films A TV Game Documentary Episode At LAFS

Earlier this month I received an email from Mediajuice Studios, the production company that made the documentary film Video Games: The Movie, which was released last year in theaters, iTunes, and Netflix. Mediajuice was currently in production on a documentary series titled Unlocked: The World of Games Revealed, an original series on the video game industry and community, and were asking permission to film a segment about The Los Angeles Film School’s Game Program.

Being constantly on the lookout for ways to promote our game program, I lost no time in passing on this request to my boss, our Vice President of Education, as well as our Vice President of Marketing. I was starting to get worried when almost two weeks had passed without hearing back on this, when I received an email from our Marketing & Events Manager at 4pm on a Friday confirming that the film crew would be here on the following Wednesday morning, and they wanted to have faculty, students and alumni available to be interviewed by their celebrity correspondent (and the show’s executive producer), Sean Astin.

This news was especially exciting for a couple of reasons. First, I was a big fan of The Lord of the Rings films in which Sean Astin starred as the faithful Hobbit Sam Gamgee. The Lord of the Rings is my favorite series of books, and when I first read that Peter Jackson was planning to film this “unfilmable” story, I read every news article I could about the movie’s production. I became such an expert about how the books were being adapted into a movie script that I began posting corrections to false rumors on message boards and websites throughout the internet. I eventually compiled all of my notes into what I called Ancalagon The Black’s Complete List of Film Changes, causing me to become the internet’s biggest authority in how the films were adapting the books. For a period I started receiving more press about my work reporting on Lord of the Rings movie news, including being featured in a Wired magazine article about Tolkien fandom) than I did for the videogames I made.

This all lead me to being “hired” as chief news reporter for Tolkien Online, the internet’s most popular Tolkien message board. Each day I would search the internet for stories about the films’ production and then synopsize them for Tolkien Online (this was all while I was also working at my real job as Development Director for the Heroes of Might & Magic franchise). Once or twice I had occasion to correspond with the films’ stars, but the high point of my “journalism” career was having a picture taken of my oldest son, Ben, and I with one of the film’s stars, Sean Astin, at a 2001 Lord of the Rings book signing I was covering in Beverly Hills. And now, fourteen years later, I had an opportunity to meet our favorite hobbit again.

However, the news that Sean Astin would shortly be coming to do an episode about our program was also exciting in a completely different way: there was preciously little time available to wrangle faculty, students, and alumni to be interviewed. Now, two business days might seem like more than enough time to gather up interview candidates, but the challenge is that our students (and faculty) are only in class every other day. We are a professional school, and we operate on a very different schedule than most colleges and universities: our terms are four weeks long and our students usually take only one class during that term. Classes can also be scheduled during the day (8:30am to 3:30pm, with an hour break) or the evening (4:30pm to 11:30pm). With the video crew being here only on one day, from 10am to 2pm, the chances are that most students and faculty I wanted to be interviewed would either have to come in on a day they wouldn’t normally be on campus, or would have to come in six hours earlier than normal.

With no time to lose, I began contacting alumni over the weekend. I first reached out to Christopher Federici, one of our star alumni. I first met Chris about four years ago when I was a producer at Jet Morgan Games and in need of a Flash programmer. I was also on The Los Angeles Film School’s Game Program Advisory Committee, and when I asked the Program’s then-director, Bob Bryant, if he had any students who were good at Flash programming and would be graduating soon. Bob suggested that I attend the monthly Game Fair to search for talent. This is where I met Chris, and after an interview at work, I hired him as my programmer. Within a couple of months Chris was promoted to associate producer and now he is working as a Technical Coordinator at Riot Games. The Unlocked crew said that they were looking for stories to tell through the interview, and I thought Chris’ story would make for a good one. However, after trying for several days to contact Chris my email, text message, Facebook and LinkedIn, I learned that he was away for the week, visiting his parents in New York.

Fortunately, I had another superstar alumni up my sleeve. John Doherty was a member of Chris’ LA Film School graduating class whom Chris suggest we interview when we had another associate producer position opening at Jet Morgan. I remember at the time grilling John for over an hour with questions about game production, and when I couldn’t stump him, I thought, “Wow, LA Film School must have an excellent program!” We did hire John, and not only was he an excellent producer for our company, he went on to become an associate producer at Disney Interactive. I thought this would be another good story for the Unlocked program, especially since I was the very first game producer ever hired at The Walt Disney Company and there was sort of a symmetry there. It turned out I was able to reach John very quickly, and he said he didn’t think he’d have problem taking time off from work on Wednesday for an interview.

On Monday, I met with our Career Advisor, Kevin Bannerman, to see if he could contact other alumni to show up on Monday. The Unlocked crew wanted to film scenes in our Computer Lab, which is where many of our Game Program classes are held, but the mandate from my management was that the filming couldn’t disrupt any scheduled classes. It turned out that a Game Audio class was scheduled to be in the Game Lab that day, but at 11:30am they would be moving down the street to work in a recording studio at our Recording School building, leaving the Game Lab available for taping the episode. My idea was that we would stage a “mock classroom” in the Lab so that it wouldn’t look empty, and I needed Kevin’s help finding students to fill up the room.

We especially wanted alumni who had game projects to show off to appear, since the Unlocked crew wrote that they were interested in seeing student projects. We contacted one recent student who had made a very nice 3D mobile game using Unity as his final project and managed to get it published through the App store, but unfortunately, he had an important deadline to meet at the game studio he was now working at and couldn’t take Wednesday off. We had better luck with two other graduates who had together developed another 3D Unity game using the Oculus Rift as their final project: one member promised Kevin that he would be there on Wednesday, while another member told me that “I’d probably be interested in coming.”

Probably be interested?! In being interviewed by a famous celebrity on a television show that might help your career? Argh! How many times have I told my students that to get a job in the competitive game industry, you have to network, network, network, and seize every opportunity to get noticed! I was amazed that the graduate didn’t jump at this opportunity, but I soon found the lack of interest was disappointingly pervasive among so many of the people I tried to get to participate.

Kevin tried contacting a number of other alumni to show up for the taping, but most couldn’t get off work that day. We were concerned that none of our female students were available to be on the episode. I suggested that we try to bring in three young women who had worked together on the previous month’s Final Project, but none of them returned Kevin’s calls. The Final Project that was being developed this month was conceived and being programmed by one of our best students, Krystal, but she was out sick. When I finally reached her Tuesday evening, she said that she too was working on Wednesday and couldn’t take the day off.

I did manage to get commitments from two of our current students who I thought would be good interview subjects: Nick, who was interning for Atari founder Nolan Bushnell’s latest company, Brain Rush; and Jerry, a new student who I found had an encyclopedic knowledge of game history and created several games in GameMaker on his own before enrolling in our program.

Now to get commitments from faculty. I didn’t know if the Unlocked crew wanted to interview people individually or do so as a group, and so I tried to get as many people to commit as possible. I called together a special meeting of our game faculty at 3:30pm on Monday to see who would be interested in attending the filming. Unfortunately, only two Game instructors were on campus and showed up the meeting — Mike Dawson, our programming instructor, and Karen McMullan, our level design instructor. Although Mike would not be teaching on Wednesday, he would be available. As for Karen, her class wasn’t scheduled until 4:30pm on Wednesday but she would get her class to come in early at noon; however, she didn’t want to appear on camera herself. As for the other instructors, I’d have to hit them up tomorrow about their availability for filming.

I spent most of Tuesday trying to reach people we could contact and verifying commitments with the ones we had. The person I most wanted to get a commitment from was John Doherty, and it wasn’t until Tuesday afternoon after I sent him a reminder via Facebook that he managed to get out of a meeting he had scheduled for Wednesday so that he’d be available to come to campus. Just before teaching my own class at 4:30pm, I glanced at that fourteen-year-old photo of Sean Astin with me and my son Ben, when it occurred to me that my other son, Timothy, was actually the family’s biggest Lord of the Rings fan, and I see if I could get Sean to take a picture with him too. During a class break, I called my wife and she agreed to let Timothy get out of school tomorrow and attend the film.

As luck would have it, the freeways were unusually jammed on Wednesday morning. The Unlocked crew was supposed to arrive at 10am, and so I had planned to come to my office an hour earlier to print out materials for them. But with this traffic, I didn’t reach my office until 9:40am. As soon as I sat down and turned on my laptop, I got an email from Jeremy Snead, the producer, that the crew was down in the lobby and wanted someone to meet them. I told my son Timothy to wait in the office while I ran down to the lobby.

After greeting Jeremy, I handed him a list of the faculty, students and alumni who agreed to be there today and why I thought they would be interesting to interview. Sean Astin hadn’t arrived yet, but the plan was for Sean to do an interview with me in the lobby, and then take a short tour of the campus with one of our tour guides, Whitney. As I was waiting for the film crew to set up in the lobby, Whitney asked me what she should say about our Game Program, because she was much more acquainted with our school’s other programs. So, I dashed back to my office to print out a copy of script for the orientation tour that we give to incoming Game students.

When I got back down to the lobby, I saw that Sean Astin had arrived and was talking with some of the production people. Just then, one of our Animation instructors, Rob Silva, began setting up equipment in the lobby for an event he had scheduled at noon. Sean then went over to help him out, and somehow wound interviewing Rob instead. However, everything seemed to work out, as Rob had an Oculus Rift headset with him, and the crew filmed Sean trying it out. Meanwhile, I set a text to my son Timothy to come down to the office right way.

As the crew began to set up for their shots of the tour with Whitney, Sean came over an introduced himself to me and we discussed, off camera, what I teach at the school. Without trying to sound too much like a fanboy, I told him about my own experience with Lord of the Rings, and how for a brief time, I was more famous for my Rings reporting than I was for my work in game development. Sean then very enthusiastically agreed to take a picture with me and my son Timothy, and seemed genuinely tickled to see the photo he took with me and my other son, Ben, fourteen years ago. (Timothy later told me, “That was cool” — I got big Dad Points for setting that up).

Sean then got called away to shoot his scenes with Whitney, and so I went into the Computer Lab to set things up. Working with producer Jeremy Snead, we put Nick, Jerry, and John in strategic positions for being interviewed when Sean entered the room. (I also put myself and Timothy in good background positions so that we would be included in some of the shots).

When Sean went into the Lab, he immediately went over to the teacher’s desk to interview Amy Zimmitti, our Game Audio instructor, who had remained for short while after her class had ended. Ironically enough, Sean then moved over to talk to Karen, the Level Design instructor who originally did not want to appear on camera, but she gave a very enthusiastic and informative interview. Plus, I was happy that Sean interviewed our two female instructors because all the students in the Lab were guys, and I didn’t want this to be another case of women being underrepresented in the game industry.

Filming concluded with a talk with our superstar alumni John Doherty. As I told John afterwards, he could not have handled the interview better than if I had written a script for what I wanted him to say about our Game Program. John eloquently described the value of earning a Game Development degree at our school (my favorite part was when Sean asked “Is it worth the price of a degree here?” and John earnestly replied, “It would be worth it at any price”), and Sean seemed completely enthralled by him. In fact, they were only supposed to talk for two minutes, but the interview went on for about twenty. I think the Unlocked crew finally found the story they wanted to tell.

Afterwards, as the crew was packing up, one of the students I requested to be there, Jerry, asked me if he was still going to be interviewed. He looked very disappointed to hear that there wasn’t enough time after all (filming was supposed to last to 2pm, but they needed to leave an hour early). I sympathized with him; I missed my time on camera too. But, as I told him, keep on making newsworthy achievements, and more opportunities will come to you.

As soon as I found out when the episode airs, I’ll post it about it on my blog.