Further Project Proposal Work

I had a productive meeting with Dr Dave Ellison last week were we looked at my draft introduction and I was given good feedback which suggests that I have a good understanding of the topic and the requirements of the proposal.

So my plan for this week is to have a finished draft proposal and arrange another meeting with Dr Ellison so that I can get some idea of how markers will look at it and any areas that need improvement before I have to submit the final copy on the 17th.

Introduction PP

Well it's getting near the end of term now so I've started on my project proposal, what I've done so far is the introduction but I'm still not quite sure what is supposed to be in it so it's possible some of it may have to be moved out into the abstract.

• Introduction

· AI is becoming more important in games development

The games industry used to be primarily focused with showing off new graphics techniques as soon as they were available and as such much of the development cycle was given to the area of a games graphics. However now the Artificial Intelligence (AI) of a game is becoming of more increasing importance as developers and publishers alike realise that game AI, like high end graphics, has the ability to make games seem more realistic and therefore increase its playability (Nareyek 2004) therefore giving a title a unique selling point.

· Evolutionary AI helps to increase enjoyment in a game

Part of AI that is widely overlooked as a means of judging efficient game AI (Rabin 2004) is evolutionary AI, which is the ability of AI to evolve to a higher standard based upon the strongest elements of its previous implementation. Large online communities surrounding a game are becoming more and more common place as more games implement multiplayer capabilities which allow players to play against opponents from anywhere in the world, this places extra demand on developers to develop AI that is of the highest standard so that players are still challenged by opponents that they have faced multiple times, to keep up with the ability of games to offer unique opponents with every game. There are a lot of different methods for implementing learning into AI, however some techniques fail to meet the necessary requirements to be implemented into a commercial computer game and therefore learning AI is treated with a cautious scepticism by developers (Manslow 2002). This is very unbeneficial as evolutionary AI has the ability to be implemented into most genres of computer games and is therefore an area that should be tackled at the highest levels of academia and professional development is resolving the issues that prevent its implementation into commercial games.

• Talk about dynamic scripting

Dynamic scripting is a relatively new method for learning AI that was introduced to be an effective form of reinforced learning for commercial computer games and meets the requirements that currently other forms of learning AI fails too (Spronck and Kuyper and Postma 2004). Dynamic scripting works by using domain knowledge from a rules base, which is populated with predefined rules, and selects rules based upon a weights values and inserts them into the scripts which control AI behaviour. The weights of the rules are calculated based upon a particular rules performance in its last implementation against the human player and the weights are increased or decreased depending upon the success or failure which leads to rules with higher weight values being more likely to be selected (Spronck and Kuyper and Postma 2004).

• Research Question

“Is Dynamic Scripting an effective method of online learning for game AI in RTS games?”

• RTS games use tactics that need to be able to adapt to how the player is playing the game

The reason for testing the implementation of dynamic scripting into real time strategy (RTS) games is, since RTS simulations require players to be able to replay scenarios, this is a term used to describe a RTS match, which are the same in principle, access to the same buildings and troops, but require players to adapt their tactics based upon opponents behaviour, the size of map being played on and the amount of resources available. Therefore for a commercial game to remain challenging and entertaining to a player it is necessary for the computer controlled opponents to adapt to human players tactics, as the common practice for most game players is once they have a tactic that leads to victory they will rarely change it, until it is no longer an effective method of attaining victory.

• Proposal Structure

The proposal structure is still empty because I'm aranging a meeting to see my project supivisor, Dr Ellision, to disscuss whats soposed to be in the proposal, but I'm also going to try taking to Dr Fortuna as well since he has had more experiance marking project proposals.

Will update again after talking to my supivisor,

Later

Stuart

RQ Presentation

Well this week we had a ethic lecture in space of our usual dissertation one, fortunately I think mines should fly past no problem because I don't plan on involving anyone with my project, i.e in the form of a questionnaire or testers, and there are no ethical concerns I can think of with computer AI.

I also had a meeting with Dr Dave Ellison last week about my topic as he is my new technical supervisor and he seems fine with my topic as well, basically saying the same as Dr King when I meet with him a few times.

Mostly this week apart from reading Dr Pieter Spronck's Phdis, which is a highly insitful document into game AI and the dynamic scripting technique, I have also been getting ready for the RQ presentations we have to give tommorow. Mines in the afternoon so hopefully going to get another practice session in in the morning (already had one with Stu, Joe, Dave anf Jimmy last week) and I'm looking forward to hearing other lectures views on my topic.

Stuart "Tsoski" Robb

Worksheet 4

This is a copy of the final worksheet for the planning module for my honours dissertation project.

Introduction

The aim of this project is too look at dynamic scripting and try to determine what would be the most suitable method of implementation for use in a commercial real time strategy game. Dynamic scripting can be used in games were the artificial intelligence is defined in scripts, or rules, which are executed sequentially. What dynamic scripting does to this method of AI implementation is, it applies a weight value to each of the scripts, or rules, which can be updated so that rules with higher weight values, which are considered to be more successful than those with lower values, are more likely to be selected for use by the game AI. For instance the project will look at offline learning, which is dynamic scripting which calculates the new weight values for the rules after a game has been completed rather than during the game. As well as the underlying method for the learning algorithms for the AI for example, natural networks, genetic algorithms and evolutionary learning and try to determine the most suitable method of AI judged against factors that are important to commercial game developers.

Issues

The issues that this project is going to address are why dynamic scripting is not already implemented in commercial games and the reasons why it is more preferable to the current AI algorithms already being used; including the issue that using dynamic scripting can lead to inferior AI especially if the player deliberately misleads the AI. This problem was looked at and a method of history fallback, were the weights associated with specific rules are reverted to a previous score when the AI tarts to produce less than desirable results, however the results of this method didn’t produce any significant advantage and therefore requires more optimisation. The method of automatic rule ordering for dynamic scripting has been shown to be just as effective as manual tuning rules (Spronck et al 2007) therefore automatic rule ordering combined with a means of history fall back could help prevent undesirable AI behaviour.

Research Question

"Is Dynamic Scripting a suitable method for AI decision making in a real-time strategy game?"

Addressing The Question

To address the question and to show significant results I will be using an open source game called “Wargus” which is a clone of the popular “Blizzard” real time strategy game “Warcraft 2” and I will try some different multiple implementations of dynamic scripting with a different underlying algorithm for the AI of the opponent in the game. Then using a serious of tests in which the game will be played by two AI controlled players one using the original implementation of the game AI and the other controlled by one of the different implementations of dynamic scripting I should be able to show which of the implementations would be best used in a commercial game using a few of that factors that are important to games developers i.e. Fast, Effective, Robust and Efficient (Spronck, Sprinkhuizen-Kuyper & Postma 2004)

Progress

So far the progress that has been made for the project is the selection of a suitable arena for the testing of dynamic scripting, i.e. “Wargus” which is an open-source clone of Warcraft 2 and I have acquired a “lua” editor, lua being the scripting language used by the engine which is the foundation of “Wargus”. I have also started to read tutorials on “lua” and I am starting to become familiar with its implementation and will then start to try editing the “Wargus” AI with some simple example to begin with. I have also read numerous articles and journals, most co written by Pieter Spronck who first suggested the idea of dynamic scripting, which discuss the issues associated with dynamic scripting and met with Dr King, a lecture at the university, twice to discuss my project and have arranged to meet my new project supervisor so that I can have his opinion on my topic as well.

Worksheet 3

Here is a copy of the worksheet 3 which is a draft copy of my project proposal.

Introduction

1.

The aim of this project is too look at dynamic scripting and try to determine what would be the most suitable method of implementation for use in a commercial real time strategy game. For instance the project will look at offline learning, as well as reinforced learning, evolutionary algorithms and natural networks and try to determine the most suitable combination judged against factors that are important to commercial game developers.

Motivation

For most games, challenging gameplay is equivalent to having high-quality game AI (Laird 2000). Therefore commercial game developers are trying to make sure that there games are using high end AI, and dynamic scripting is an effective method to improve game AI. I think that it’s also important to have a strong AI opponent in games as a human opponent might not always be available and the current level of AI in games is too easy and static so it becomes less challenging and boring over time. Currently the research that has been done into applying dynamic scripting in games has looked at improving the implementation of dynamic scripting (Spronck, Sprinkhuizen-Kuyper & Postma 2004) and the research has already decided upon using an offline form of evolutionary learning, where as I think it may be possible to try using online learning with different forms of adaptive AI to improve the underlying game AI.

Research Question

"Is Dynamic Scripting a suitable method for AI decision making in a real-time strategy game?"

Addressing The Question

To address the question and to show significant results I will be using an open source game called “Wargus” which is a clone of the popular “Blizzard” real time strategy game “Warcraft 2” and I will try some different multiple implementations of dynamic scripting with a different underlying algorithm for the AI of the opponent in the game. Then using a serious of tests in which the game will be played by two AI controlled players one using the original implementation of the game AI and the other controlled by one of the different implementations of dynamic scripting I should be able to show which of the implementations would be best used in a commercial game using a few of that factors that are important to games developers i.e. Fast, Effective, Robust and Efficient (Spronck, Sprinkhuizen-Kuyper & Postma 2004)

Resource Requirements

For the project I am going to use an open source game called “Wargus” which has as its underlying game engine “Stratagus” which is an open source real time strategy engine, and I will be editing the scripts for the AI in “Wargus” which are writing in the scripting language “Lua”. To do this all I will require is a computer with “Wargus” installed and a Lua editor.

References And Bibliography

Bourg, D.M. and Seemann, G. 2004. AI for Game Developers. [online] Available from: http://proquest.safaribooksonline.com/0596005555 [Accessed 07 October 2008]

Ponsen, M.J.V. and Muñoz-Avila, H. And Spronck, P. And Aha, D.W. 2005. Automatically Aquiring Domain Knowledgw For Adaptive Game AI Using Evolutionary Learning. USA: American Association for Artificial Intelligence.

Ponsen, M and Spronck, P. 2004. Improving Adaptive Game AI with Evolutionary Learning. In: University of Wolverhampton.

Rabin, S (ed). 2004. AI Game Programming Wisdom 2. Massachusetts: Charles River Media, Inc.

Spronck, P. and Sprinkhuizen-Kuyper, I. and Postma, E. 2004. Enhancing the Performance of Dynamic Scripting in Computer Games. The Netherlands: Universiteit Maastricht, Institute for Knowledge and Agent Technology.

Update

Well this posts a little later than expected, but that just because we missed a lecture 2 weeks ago so that we could brush up on our library skills so we've still not caught up and been shown how to form our proposal yet.

But in the mean time I've still been reading over a couple of articles most featuring Dr Pieter Spronck who was the man who spearheaded "dynamic scripting" and form the research that hes already done it keeps me thinking about where I want to go with the project and so I find my question under fire again.

For now I can't decide if I want to continue with comparing "dynamic scripting" against other forms of learning AI such as "neutral networks, evolutionary algorithms and reinforcement learning", actually I'm not to hot on these methods yet but they've been mentioned in a few of the articles as examples of inferior AI. Or whether to continue to try improve "dynamic scripting" by trying to reduce the length of time it takes for the "dynamic scripting" method to get better, i.e. win over normal game AI.

Another change to my project involves that I have been able to successfully get "Wargus" working which I now intend to use instead of "ORTS" as since its a clone of warcraft 2 effectively, it means that it will be easier to develop a rules base using a strategy guide or forum.

Till Later
Fellin' Fine

Worksheet 2

Well this week involved the completion of worksheet 2, which was submitted to Dr Özveren on Wednesday after being given a look over by Dave, who added his comments to it before passing it along.

Basically this worksheet was to get us thinking about a research question (RQ) as well as getting us to think about how we were going to write up our proposal, which I believe is due to be presented in the next couple of weeks sometime. So what I have as my RQ at the moment is:

What are the advantages and disadvantages, in relation to difficulty level of the computer opponent, ease of implementation and cost to the CPU, to using evolutionary learning as the foundation of the artificial intelligence in a real time strategy game compared to the more traditional scripting methods for real time strategy computer controlled opponents?

I have just had a second meeting with Dr King, and he has suggested that this is a bit long for a RQ and i would be better to shorten it using just the key words, advantages and disadvantages, evolutionary learning, real time strategy, and then in my proposal go into some more detail about what I hope to achieve and look at during my project.

The rest of worksheet 2 was about how the RQ would be implemented and I won't go into to much detail about that just now as based on Dr Kings feedback it's likely that it will be very similar, if not the same, to how I research my dissertation so more will be talked about it as I do it. However I will say that the plan for the moment it to look more into "dynamic scripting" as I think this will be the algorithm of evolutionary learning that I hope to look at and try find possibly another method to look at, as Dr King suggested that I may just want to wait see how time permits before turning the dissertation into a comparative study.

I am still planning on using the ORTS engine however, I have seen 2 separate journals that used the Stargus engine and the only reason I decided not to go and use that one is for the reason that I don't know Linux to well and so I would essentially have to learn two languages at once just to run the experiments which, I could just run through a simple program rather than a game if time constraints get to large, however I would really like to have my algorithm implemented into a game for the end of year presentations.

So my next blog update should hopefully be a bit earlier in the week as I am going to send a copy of my proposal, if the information is up on what I need to write for it, to Dr King before having to submit my final copy.

So until then, Feelin' Fine
Stuart "Should have learned linux as a child" Robb

Preliminary Ideas.

Well It's been a lot longer than I expected between blog post but at least I've got a bit to talk about what I've done in the last week.

I managed to decided upon which of the two topics I'm going to look at, I've decided to go with my third option partly based upon the feedback I received from the first Worksheet I submitted to Dr Özveren but also because of the articles I read over the weekend on "Learning AI". Learning AI does come under many different names, "Evolutionary AI", "Adaptive AI" etc, so I think a bit more explanation is required.

What I intend too look at, is the process of computer opponents "learning" how to play the game better, in the same way that human players do. I'm plan on looking specifically at how this would work in a Real-Time Strategy (RTS) game and will base my experiment on this. I read a fantastic article on the subject called "Improving Adaptive Game AI With Evolutionary Learning" (http://www.cs.unimaas.nl/p.spronck/Pubs/PonsenCGAIDE.pdf) and this is what gave me the idea for my project.

I have just come back from a meeting with Dr David King, who has been proposed as my project supervisor at this stage, and after that meeting were we discussed my idea, where I would want to go with it, and he has suggested since I have until next Sunday to form a "Research Question", the question which my dissertation is intended to answer, I should just keep doing what I'm doing and read some more around the subject area and think about some specific "learning" techniques to look at as the whole area of "Learning AI" may be a bit too broad a subject area.

I have also found an open-source RTS engine developed by the University of Alberta called ORTS, Open Real Time Strategy, this engine is used by the University in an annual AI competition they run and so i feel it should be a good starting point for running experiments to test "Learning AI" algorithms on as there is an active development community, and since it is used specifically for AI competitions it should be able to accept different techniques.

So my plan for now is to go to the library, or have a look at some of the online databases, for copy's of the books mentioned in the "Improving Adaptive Game AI With Evolutionary Learning" article and continue to work on ORTS and make sure that it will be easy enough to use for my experiments.

Till Later,

Stuart
Feelin' Fine

Getting Started!

Good evening,

This is a "Blog" that I have set up to help me, my tutor, classmates and anyone else that takes an interest keep track of my progress over the next 7 months as I prepare too and write my final year Honours Dissertation.

I suppose the best place to start would be to give a little background info, I've just started my final year at Abertay University studying a BSc in Computer Games Technology, a course that I hope will help lead to a programming job in the computer games industry. Part of your final year, as I'm sure many people reading this blog will be aware off, involves writing a Dissertation on a topic of your choice in which you are required to do most, if not all, of the research and writing of yourself.

Well for my dissertation I've decided upon basing it on Artificial Intelligence (AI), you are probably thinking this is a bit of a large topic area and well you'd be right and so I am currently working on narrowing that scope down into one of three areas, which i specified in the first "Worksheet" I was asked to complete as part of my dissertation module at university:

1. Looking at the difference between “rules based” and “script based” forms of AI, and how rules are generally considered to be more realistic and challenging in games however scripting is a lot faster to execute in a game and why it is the method normally implemented.

2. Looking at what would be the most effective method for calculating for AI for hunting a target in 3-Dimensions, e.g. in a flight combat game. Examples of areas to look at would be ray-tracing or A* path finding.

3. Or learning/adaptive AI for example in a strategy game, Command & Conquer for example, rather than scripting the computer to build a base, following a set pattern every match, looking at ways for it to develop its own strategies to combat a human opponent as it gets to “know” the players tactics. In the same way as players online would get to understand other players methods the more they played each other.

Out of these three more specific areas I find myself currently leaning in the direction of No.1 as I have read a paper on this area already and see that there is a lot of interest on both sides of the argument, however I also feel that No.3 could be quite interesting and after reading an article on "Gamasutra" (http://www.gamasutra.com/features/20041122/bourg_01.shtml) I have decided to give it some more consideration.

So my plan for the weekend now is to read up more on these two areas, and read over the feedback from my module tutor, Dr C Süheyl Özveren, which should arrive by Friday and I should have decided by the beginning of next week, when I'll start formulating my "Research Question" more on that next week!

Feelin' Fine!
Stuart "Tsoski" Robb