\documentclass[letterpaper,10pt]{article}

\usepackage{titlesec}
\usepackage[margin=0.5in]{geometry}
\usepackage{graphicx}

\titleformat{\section}{\large\bfseries}{\thesection}{1em}{}

\begin{document}
\pagestyle{empty}

\title{Adaptive Mario : CS8803 Game AI Project 3}
\author{Woody Folsom and Marshall Gillson}
\date{March 18, 2012}

\maketitle

\section{Introduction}
The game mechanics of the Super Mario world are celebrated and well-understood. Indeed, even in this age of super-computer graphics and Hollywood-level production values, few games can match the pure enjoyment delivered by this simple 8-bit Nintendo platform jumper.  Here, we endeavor not to change that formula, but to imbue it with a modern, artificial-intelligence driven approach to enhance the game-play experience.
 
In creating Adaptive Mario, we use several procedural content generation (PCG) techniques built upon this simple platform-jumper engine. Our algorithms adjust to player preferences and skills, and guide players toward an ideal game-play experience. Our design is intended to ensure that the maximum amount of replay value is delivered, considering the limited multimedia assets and game-play mechanics of the Infinite Mario engine on which it is based.
 
Every Mario level has the implicit goal of progressing from left to right until the completion gate appears, all the while keeping Mario safe.  A number of more esoteric subgoals also arise, such as destroying enemies, collecting coins, and minimizing time of completion. Our conceptual design goal, however, was a bit different.  As shown in Figure \ref{mario-ex} we aim to provide players with different styles of play and skill levels with distinct and customized game-play experience from the moment the level begins. The process of generating a level consists of a number of steps. First, a player profile is drawn based on the player’s performance in an average, non-customized level. That data is fed into our generation engine to guide the creation of subsequent levels.  The level generation pipe-line is explained in full detail in \emph{Section 3 Approach}.

\begin{figure}[h!]
\centering
\includegraphics[width=0.4 \textwidth]{mario_example.png}
\includegraphics[width=0.4 \textwidth]{mario_underground.png}
\caption{Adaptive Mario in action}
\label{img:mario-ex}
\end{figure}

As shown in Figure \ref{img:mario-ex}, the conceptual goal is that players with different play styles and skill levels should have a very different game-play experience from the moment the level begins.  The very first time Adaptive Mario is run in a new environment, a random level is presented based on an a priori model of an 'average' player.  A high degree of randomness at this point allows the player to freely choose a game-play style rather than being constrained by a prefabricated environment.

\section{Related Works}
A fundamental concept in the design of Adaptive Mario is \emph{flow}. As defined by Mihály Csíkszentmihályi, according to Salen and Zimmerman \cite{zimmerman}, flow is a state of total immersion and concentration in which the player believes he or she is overcoming obstacles by the narrowest margins. As shown in Figure \ref{img:flow-state}, achieving this state involves a delicate balance between the difficulty of the game and the degree of the player's skill. If the game is too hard, the player will become frustrated. On the other hand, most players will become bored if the game is too easy. Our overarching goal was to facilitate a flow state by giving the player sufficient challenge and variety without rapidly becoming too difficult.

\begin{figure}%[h!]
\centering
\includegraphics[width=0.5 \textwidth]{flow-state.png}
\caption{Flow State: Game Difficulty vs. Player Skill}
\label{img:flow-state}
\end{figure}
 
This theory presents two variables: challenge and skill. Adapting the difficulty of a level is well within our purview as game designers, but the level of difficulty required to generate flow state depends on the player’s skill level. We therefore first developed a model of player psychology and ability. Numerous such models have been proposed since Richard Bartle's 1996 study of MUD (Multi-User Dungeon) players \cite{bartle}, all of which attempt to classify a player into a category or type. In developing our model, we considered some notable examples:

\begin{table}[ht]
\centering
\begin{tabular}{ | l | l | }
\hline
Author & Player Model\\ \hline
Richard Bartle & Killer, Socializer, Achiever, Explorer\\ \hline
Nick Yee & Relationships, Immersion, Grief, Achievement, Leadership \\ \hline
John Radoff & Immersion, Cooperation, Achievement, Competition \\ \hline
\end{tabular}
\end{table}

The Adaptive Mario PlayerProfileMatcher class assigns the player one of five roles based on past performance: BUMPER, COLLECTOR, JUMPER, RUNNER or SHOOTER.  This model is loosely based on Bartle's extended model, with BUMPER corresponding to Planner, in the sense that the player is observed to interact with the environment by bumping bricks and throwing shells.  However, our model necessarily differs somewhat from the above examples, as Adaptive Mario is a single-player game with no social element.

\section{Approach}
The Adaptive Mario level generator functions as a 4-stage pipeline composed of a Profile Matcher, Level Archetype Selector, Level Generator and Challenge Component Creator.

\subsection*{Profile Matcher}
The first step of the level generation process is to evaluate the player's style and skill level. Adaptive Mario uses the output file ``player.txt'' generated by the GamePlay class during the most recent game, if available. If no player data is available, as in a first run of the game, a simple, un-weighted level is generated, to assist in collecting baseline player data for future runs. More detailed statistics from the DataRecorder class are preserved as XML and are used by some components of the level generator. The ProfileMatcher scores aspects of game-play such as number of kills, jumps, and coins collected in order to evaluate the player's skill in the five identified areas of competency and generate a vector of skill values, within a range of 1 -- 100. The relative weight of each different skill is maintained by the player profile for reference by later segments of the level creation process. Here, the player is also assigned a type based on his or her most dominant trait, and an overall skill level: expert, proficient, competent, beginner, or novice. A player’s skill level is based on a number of different factors. First of all, his or her average skill level across the skill vector  is included. The speed with which the previous level was completed is integrated, as a player who finds a level too easy can complete it quickly and without error. Finally, the number of times their power-up state changed; if a player is not being threatened or hurt will tend to remain in one power-up state through much of a level, with few changes.  A weighted overall skill score is produced and mapped to one of the five skill levels  as shown below.

\begin{table}[ht]
\centering
\begin{tabular}{ | l | l | l | }
\hline
Score & Skill Level & Attributes\\ \hline
20\% & Novice & Low situational awareness, Reflexive responses\\ \hline
40\% & Beginner & Uses judgment to react to challenges, Limited awareness \\ \hline
60\% & Competent & Copes with multiple challenges, Uses sound strategy \\ \hline
80\% & Proficient & Makes rapid decisions, Prioritizes goals \\ \hline
100\% & Expert & Intuitively solves challenges, Pushes limits \\ \hline
\end{tabular}
\end{table}

These categories are based on the Dreyfus model of skill acquisition with each successive level representing an achievement of 20\% of the Profile's key skill maximum score, before penalties.  Actions such as dying or making unnecessary jumps negatively impact the skill assessment.

\subsection*{Level Archetype Selection}
The Level Archetype selector picks an overall macro-form for the level based on the player's profile and skill level.  As there are five categories and only three distinct environments in the Inifinite Mario engine (Overland, Underground and Castle), different profiles cause similar Level Archetypes to be generated.  However, this does provide the opportunity to indicate to the player what types of challenges lie ahead and may serve to enhance immersion.

Next, the Level Archetype is further customized for the player by disabling challenges and enabling rewards according to a predefined set of rules.  In order to allow the rule set to be easily customized and extended, Adaptive Mario uses Drools, an implementation of Charles Forgy's 1982 Rete algorithm \cite{forgy}.  Once the appropriate rules have fired, level generation proceeds to the third stage: macro-structure generation.

\subsection*{Level Generation: Macro-structure}
The task of the macro-structure generator is to deliver a list of sequential Mario level design elements (LevelComponents) based on a high level description similar to ``Overland level with gaps but no pipes, difficulty 5/10''.  Several algorithms were considered during the planning phase of this module.  A Genetic Algorithm approach could potentially produce novel levels, but judging the relative fitness of candidate levels could be problematic.  Hierarchical task networks (HTNs) and rhythm-based approaches have also been used, but tend to generate levels with a very uniform structure in a single pass.  For ease of modification and to leverage the authors' design intuition, a context-free grammar (CFG) was implemented instead.

Given a suitable array of predefined LevelComponent types, each of which corresponds to one or more Infinite Mario game sprites, the structure of a level archetype can be completely specified in the form of a file-based CFG.  As shown in the following example, the OR clauses are assigned probability values, giving a stochastic property to the generated level structure.

\begin{figure}[h!]
\centering
\includegraphics[width=0.75 \textwidth]{StochasticGrammar.png}
\caption{A Simple Overland Level Grammar}
\label{img:stochastic-grammar}
\end{figure}

One potential pitfall of using a CFG for this purpose is that the generated level may be over-specified (containing too many elements) and hence too crowded, or under-specified and nearly empty.  To mitigate this problem, a fitness evaluation function iteratively invokes the LevelGrammar class's generateRandomTree() method, rejecting proposed levels with too many or too few LevelComponents.

\subsection*{Challenge Components: Micro-structure}
Challenge components are the building blocks of an Adaptive Mario level. Each challenge component represents a small puzzle or level element. Each level consists of a sequence of challenge components. Through only a small number of these components, we have achieved a wide variety of possible levels. In fact, the challenge components induce variety and adapt to the player’s style in the following ways.
 
First, the selection of components plays a large role in defining the character of a level. Many challenges are better suited for different play styles, based on the skills they require to traverse or the rewards they cause a player to incur. This allows a level to shift toward a more engaging landscape for a player of a particular style. Second, each challenge component dynamically adjusts its difficulty based on the player’s skill level.  The player’s skill competencies affect the number of enemies they encounter, the number of power-ups they are offered, the number of coins placed on a level, the difficulty of the enemies they face, the size and number of pits in a level, and the types of terrain encountered. Because challenge components are so dynamic, even a single challenge element encountered repeatedly can present and engaging challenge.
 
An additional benefit of challenge components is their modularity. Though we have implemented a strong suite of challenges, innumerable more are possible. With the implementation of a function and slight modification of the level generation grammar, new challenges can be added to Adaptive Mario to make it even more variable and extend its replay value further.

\begin{table}[ht]
\centering
\begin{tabular}{ | l | l | l | }
\hline
Difficulty & Challenge Component & Description \\ \hline
1 & Coin Dive & Some empty blocks, coins line the path to the ground.\\ \hline
2 & Free power-up & Sets the player up to get a power-up with little or no challenge.\\ \hline
3 & Straight & A straight stretch of land with maybe one enemy and maybe some coins or blocks\\ \hline
4 & Single Pit & A pit that the user must jump over; rocks on either side.\\ \hline
5 & Bowling Alley & A red koopa right before a long line of enemies. Kill them all!\\ \hline
6 & Cannon Line & A stack of 2 or 3 cannons.\\ \hline
7 & Maze & A maze of indestructible blocks, with enemies.\\ \hline
8 & Lemming Trap & A little pit with a few enemies that jump down into it.\\ \hline
9 & Platform Jump & A bunch of platforms to jump between.\\ \hline
10 & Pipe Jump & A bunch of thin pipes to jump between.\\ \hline
\end{tabular}
\end{table}

\section{Evaluation}
Developing Adaptive Mario presented several challenges. From a technical standpoint, finding a Java-compatible Rete rule engine became a time-consuming effort, due mostly to the large number of third-party JAR dependencies. Ultimately, we were able to incorporate the Drools engine into our code base. The amount of time, however, that was invested in doing so forced the minimization of role of the rules engine. It therefore makes only basic alterations to random levels based on the player profile.
 
Randomized level generation using a stochastic CFG was largely successful, however. Not only can our very simple grammar generate a huge variety of levels, the implementation of a parser for the rule set allows future designers to make meaningful content changes without needing to recompile source code. Such grammars are, of course, limited to some degree. The grammar by definition lacks any contextual awareness. It therefore does nothing to avoid awkward juxtaposition of challenge components. Though challenge components are self-contained such that transition between any two should not present a problem, we nonetheless recognize this stratification as a limitation of our architecture.
 
Perhaps the most challenging aspect of our development process was the creation of the PlayerProfile. Each level is generated based on player data from only the single previous level traversal. Thus, the terrain of a previous level may have an inordinate effect on the evaluation of a player’s current tastes. We built our ProfileMatcher to minimize this possibility, but it nonetheless exists. A more complex version of Adaptive Mario would benefit greatly from a more iterative and flexible approach to player modeling, perhaps by employing an artificial neural network or Bayesian inference. These techniques would allow for more accurate and therefore useful scoring values, but would require more involved modification of the game engine.

\section{Conclusion}
It seems clear from these results that the framework of Adaptive Mario has the potential to guide the player toward his or her own idealized version of a platform game, while still presenting a reasonable level of challenge.  Not only does difficulty scale in proportion to the player's performance (preventing frustration), but care is taken in the design of the level grammar to avoid repeatedly giving the player 'more of the same' (leading to boredom).  We certainly recognize many areas ripe for improvement (such as the limited level grammar), yet Adaptive Mario as currently implemented represents a robust engine. It provides tools for the adaptive content designer and new experiences for the consumer.  It certainly adds a novel twist on the genre for any platform gaming enthusiast.

\section*{Appendix A: Building the Game}
Building the Project 3 executable requires a Java SDK version 1.6+ and Apache Ant.

To build the game, execute \texttt{ant clean} followed by \texttt{ant} from the main project directory.  

\section*{Appendix B: Running the Game}

To run the game, change to the `dist' subdirectory following a successful build and execute \texttt{java -jar CS8803\_P3.jar}.

\bibliographystyle{unsrt}
\bibliography{p3refs}

\end{document}