Artificial Intelligence Research in Video Games By Jacob

Artificial Intelligence Research in Video Games By Jacob Schrum [email protected] Motivation Why do research in video games? Video games Simulated, controlled, environments Complex enough to be challenging Applications Video games and non-game simulators Robotics Beyond Insight into nature of intelligence Sufficient conditions for complex behavior Super Mario AI Competition

Goal: Create skilled Mario agent Placed in random levels International competition Victory: Entry by Robin Baumgarten Uses A* Search Start-up code: http://www.marioai.org/home Video:

https://youtu.be/DlkMs4ZHHr8 Uses perfect model of game Knows result of each action Plans ahead using model Searches for safe route to end Cons: A* requires accurate model Result is skilled, but inhuman Turing Test Invented by Alan Turing Father of Computer Science Cracked Enigma code Invented Turing machine Test of human-like intelligence Chat session with computer and human Which is which? Fool humans 50% of time to pass test Turing Test for Games

Goal: Bot for UT2004 Make it human-like Fool humans 50% of time Victory: Software: http://pogamut.cuni.cz/ BotPrize: http://botprize.org/ Video: https://youtu.be/1BdcNaexk3M UT^2: http://nn.cs.utexas.edu/?ut2 UT^2 won BotPrize 2012 By Jacob Schrum, Igor Karpov, and Risto Miikkulainen Used neuroevolution and human trace data

Cons: Made bot weaker to make it convincing Does not adjust challenge level Artificial Neural Networks Brain = network of neurons ANN = abstraction of brain Neurons organized into layers Inputs Outputs Neuroevolution Example Start With Parent Population Neuroevolution Example Start With Parent Population Evaluate and Assign Fitness

100 90 75 61 56 50 31 Neuroevolution Example Start With Parent Population Evaluate and Assign Fitness Clone, Crossover and Mutate To Get Child

Population 100 90 75 61 56 50 31 Neuroevolution Example Start With Parent Population Evaluate and Assign Fitness 100

90 75 61 56 50 31 100 120 69 99 60

83 50 Clone, Crossover and Mutate Children Are Now the New Parents Repeat Process: Fitness Evaluations As the process continues, each successive population improves performance Neuroevolution Game Neuro-Evolving Robotic Operatives (NERO) By Kenneth Stanley, Bobby Bryant, and Risto Miikkulainen Goal: Make game based on Machine Learning Player is drill sergeant

Create increasingly harder tasks for evolving bots Success: Behavior evolves in real time Interactive evolution: Player manipulates environment Evolved teams can face off Software: http://nerogame.org/ Open Source Reimplementation: https://opennero.github.io Cons: Evolved agents become specialists (e.g. snipers) Need multimodal behavior Ms. Pac-Man

Goal: Develop multimodal behavior Ms. Pac-Man requires behaviors for threat and edible ghosts Evolve modular policies Success: My dissertation under Risto Miikkulainens supervision Modular neural networks Evolution discovers when to use modules Unexpected task division discovered: luring behavior Cons: What if there are many agents? What if there are many actions?

Software (MM-NEAT): http://nn.cs.utexas.edu/?mm-neat Screen capture competition: http://dces.essex.ac.uk/staff/sml/pacman/PacManContest.html Videos: http://nn.cs.utexas.edu/?ml-pm StarCraft AI Competition Goal: Handle complexity of RTS game Unit control Path finding Build order

High-level strategy Be competitive with humans Victories: Different winner each year Many strategies Student Tournament: http://sscaitournament.com/ AIIDE Competition: https://webdocs.cs.ualberta.ca/~cdavid/starcraftaicomp/

Past Competitions: http://webdocs.cs.ualberta.ca/~cdavid/starcraftaicomp/media.shtm l Hard-coded rules Finite state machines Planning Supervised learning Probabilistic models What can AI do besides agent control? Galactic Arms Race Content Creation Goal: Evolve interesting content Insert into commercial game Success:

Space shooter Weapon behavior evolves Different firing patterns Based on user popularity Interactive evolution What about going beyond a single game? Game: http://galacticarmsrace.blogspot.com/ Video: https://youtu.be/7lBmiyGkQyg

Atari Games all of them Goal: System that can play any game Only use information human has Raw pixel data Success: Googles Deep Mind team Used Deep Neural Network Can learn any Atari 2600 game Can we get more general? Video: https://youtu.be/V1eYniJ0Rnk Code: https://sites.google.com/a/deepmind.com/dqn/ General Video Game Playing Goal: Play any game Dont know the game in advance Described in formal language

Competition: Previously unseen games Many skills needed Different tracks Competition: http://www.gvgai.net/ Explanation Video: https://youtu.be/iAaleW3ofyk Planning Learning Content Generation If these topics interest you

SCOPE you should apply to SCOPE! Summer research program at Southwestern First time CS has participated Two/Three students will do research with me Get paid for your time Application deadline: November 13th, 5pm Application Link: http://www.southwestern.edu/departments/ hhmi/scope-application/ Questions? Contact me [email protected]

Recently Viewed Presentations

  • The First World War Boys and Girls! War

    The First World War Boys and Girls! War

    Blockades In May, 1915 Germany told Americans to stay off of British ships They could/would sink them 1. Blockades Lusitania torpedoed, sinking with 1200 passengers and crew (including 128 Americans) Was eventually found to be carrying 4200 cases of ammunition...
  • Copyright  2013 Crosscutting Concepts, LLC. All Rights Reserved.

    Copyright 2013 Crosscutting Concepts, LLC. All Rights Reserved.

    The most famous incident where bite mark evidence led to a conviction, was in the case of the notorious serial killer, Ted Bundy. He was responsible for an undetermined number of murders between 1973 and 1978 and was finally tied...
  • Laws - University of Missouri-St. Louis

    Laws - University of Missouri-St. Louis

    Carr (Eastern District Court of Virginia 2003) Helen Carr sent fake e-mail messages to AOL customers in the United States and several foreign countries, advising them that they must update their credit card and personal information on file with AOL...
  • The Structure of the Earth's Interior

    The Structure of the Earth's Interior

    The Earth is composed of four different layers. The crust is the layer that you live on, and it is the most widely studied and understood. The mantle is much hotter and has the ability to flow. The outer core...
  • Cakes, Fillings and Frostings

    Cakes, Fillings and Frostings

    Cakes, Fillings and Frostings
  • Conceptual Design Using the ER Model

    Conceptual Design Using the ER Model

    Suppliers qty Departments Contract Parts Suppliers Departments deals-with Parts can-supply VS. needs * * sid name login age gpa 53666 Jones [email protected] 18 3.4 53688 Smith [email protected] 18 3.2 53650 Smith [email protected] 19 3.8 sid cid grade 53666 Carnatic101 C...
  • TO THE ORIENTATION FOR CLASS 5 2014-2015 Dr.

    TO THE ORIENTATION FOR CLASS 5 2014-2015 Dr.

    Indian School Al Wadi Al Kabir. 2014-2015 - Orientation for class 4. 5I. Mrs. Reshmine. Subjects Handled. Math, Science. Mrs. Aruna. Sharma. Hindi Teacher. Mrs ...
  • Happy Lesson on Chemical Reactions

    Happy Lesson on Chemical Reactions

    Happy Lesson on Chemical Reactions 8th Grade Forming New Substances Chemical reactions form new substances with different properties than the starting substances. Clues that a chemical reaction is taking place include formation of gas or solid, a color change, and...