Abstract

In this paper, we discuss guidelines for a reward design problem that defines when and what amount of reward should be given to the agent/s, within the context of reinforcement learning approach. We would like to take keepaway soccer as a standard task of the multiagent domain which requires skilled teamwork. The difficulties of designing reward for this task are due to its features as follows: i) since it belongs to the continuing task which has no explicit goal to achieve, it is hard to tell when reward should be given to the agent/s. ii) since it is a multiagent cooperative task, it is hard to decide what is a fair share of reward for each agent's contribution to achieve the goal. Through some experiments, we show that the reward design have a major effect on the agent's behavior, and introduce the successful reward function that makes agents perform keepaway better and more interesting than the conventional one does. Finally, we explore the relationship between `reward design' and `acquired behaviors' from the viewpoint of teamwork.