Revision with unchanged content. Reinforcement learning is an area of artificial intelligence that studies the ability of autonomous agents to improve their behavior in the absence of an informed instructor. Although reinforcement learning has achieved success in a wide range of applications, it becomes less consistent as the size of a task grows. This book attempts to improve the efficiency of reinforcement learning in realistic tasks by identifying a certain type of task structure. A task that displays this type of structure can be decomposed into a hierarchy of subtasks. Each subtask can be simplified using state abstraction so that it is much easier to solve than the original task. Reinforcement learning can be applied to produce solutions to the subtasks, and the solutions can be combined to achieve a solution to the original task. Experimental results indicate that hierarchical decomposition combined with state abstraction can significantly simplify the solution of realistic tasks. The book thus contributes to increasing the potential of reinforcement learning in realistic tasks. The book is directed towards researchers in Artificial Intelligence, but can also be used as a reference by professionals in Robotics and Autonomous Control Engineering.