Completed Theses
[PhD theses] [Master's theses] [Bachelor's theses]
PhD theses

Machine Learning for Classical Planning: Neural Network Heuristics, Online Portfolios, and State Space TopologiesPhD thesis, November 2022. Download: (PDF) (slides; PDF)
State space search solves navigation tasks and many other real world problems. Heuristic search, especially greedy bestfirst search, is one of the most successful algorithms for state space search. We improve the state of the art in heuristic search in three directions.
In Part I, we present methods to train neural networks as powerful heuristics for a given state space. We present a universal approach to generate training data using random walks from a (partial) state. We demonstrate that our heuristics trained for a specific task are often better than heuristics trained for a whole domain. We show that the performance of all trained heuristics is highly complementary. There is no clear pattern, which trained heuristic to prefer for a specific task. In general, modelbased planners still outperform planners with trained heuristics. But our approaches exceed the modelbased algorithms in the Storage domain. To our knowledge, only once before in the Spanner domain, a learningbased planner exceeded the stateoftheart modelbased planners.
A priori, it is unknown whether a heuristic, or in the more general case a planner, performs well on a task. Hence, we trained online portfolios to select the best planner for a task. Today, all online portfolios are based on handcrafted features. In Part II, we present new online portfolios based on neural networks, which receive the complete task as input, and not just a few handcrafted features. Additionally, our portfolios can reconsider their choices. Both extensions greatly improve the stateoftheart of online portfolios. Finally, we show that explainable machine learning techniques, as the alternative to neural networks, are also good online portfolios. Additionally, we present methods to improve our trust in their predictions.
Even if we select the best search algorithm, we cannot solve some tasks in reasonable time. We can speed up the search if we know how it behaves in the future. In Part III, we inspect the behavior of greedy bestfirst search with a fixed heuristic on simple tasks of a domain to learn its behavior for any task of the same domain. Once greedy bestfirst search expanded a progress state, it expands only states with lower heuristic values. We learn to identify progress states and present two methods to exploit this knowledge. Building upon this, we extract the bench transition system of a task and generalize it in such a way that we can apply it to any task of the same domain. We can use this generalized bench transition system to split a task into a sequence of simpler searches.
In all three research directions, we contribute new approaches and insights to the state of the art, and we indicate interesting topics for future work.

Search Behavior of Greedy BestFirst SearchPhD thesis, May 2019. Download: (PDF) (slides; PDF)
Greedy bestfirst search (GBFS) is a sibling of A* in the family of bestfirst statespace search algorithms. While A* is guaranteed to find optimal solutions of search problems, GBFS does not provide any guarantees but typically finds satisficing solutions more quickly than A*. A classical result of optimal bestfirst search shows that A* with admissible and consistent heuristic expands every state whose fvalue is below the optimal solution cost and no state whose fvalue is above the optimal solution cost. Theoretical results of this kind are useful for the analysis of heuristics in different search domains and for the improvement of algorithms. For satisficing algorithms a similarly clear understanding is currently lacking. We examine the search behavior of GBFS in order to make progress towards such an understanding.
We introduce the concept of highwater mark benches, which separate the search space into areas that are searched by GBFS in sequence. Highwater mark benches allow us to exactly determine the set of states that GBFS expands under at least one tiebreaking strategy. We show that benches contain craters. Once GBFS enters a crater, it has to expand every state in the crater before being able to escape.
Benches and craters allow us to characterize the bestcase and worstcase behavior of GBFS in given search instances. We show that computing the bestcase or worstcase behavior of GBFS is NPcomplete in general but can be computed in polynomial time for undirected state spaces.
We present algorithms for extracting the set of states that GBFS potentially expands and for computing the bestcase and worstcase behavior. We use the algorithms to analyze GBFS on benchmark tasks from planning competitions under a stateoftheart heuristic. Experimental results reveal interesting characteristics of the heuristic on the given tasks and demonstrate the importance of tiebreaking in GBFS.

Certifying Planning Systems: Witnesses for UnsolvabilityPhD thesis, April 2019. Download: (PDF) (slides; PDF)
Classical planning tackles the problem of finding a sequence of actions that leads from an initial state to a goal. Over the last decades, planning systems have become significantly better at answering the question whether such a sequence exists by applying a variety of techniques which have become more and more complex. As a result, it has become nearly impossible to formally analyze whether a planning system is actually correct in its answers, and we need to rely on experimental evidence.
One way to increase trust is the concept of certifying algorithms, which provide a witness which justifies their answer and can be verified independently. When a planning system finds a solution to a problem, the solution itself is a witness, and we can verify it by simply applying it. But what if the planning system claims the task is unsolvable? So far there was no principled way of verifying this claim.
This thesis contributes two approaches to create witnesses for unsolvable planning tasks. Inductive certificates are based on the idea of invariants. They argue that the initial state is part of a set of states that we cannot leave and that contains no goal state. In our second approach, we define a proof system that proves in an incremental fashion that certain states cannot be part of a solution until it has proven that either the initial state or all goal states are such states.
Both approaches are complete in the sense that a witness exists for every unsolvable planning task, and can be verified efficiently (in respect to the size of the witness) by an independent verifier if certain criteria are met. To show their applicability to stateoftheart planning techniques, we provide an extensive overview how these approaches can cover several search algorithms, heuristics and other techniques. Finally, we show with an experimental study that generating and verifying these explanations is not only theoretically possible but also practically feasible, thus making a first step towards fully certifying planning systems.

Counterexampleguided Cartesian Abstraction Refinement and Saturated Cost Partitioning for Optimal Classical PlanningPhD thesis, February 2018. Download: (PDF) (slides; PDF) (ICAPS 2020 Best Dissertation Award Talk)
Heuristic search with an admissible heuristic is one of the most prominent approaches to solving classical planning tasks optimally. In the first part of this thesis, we introduce a new family of admissible heuristics for classical planning, based on Cartesian abstractions, which we derive by counterexampleguided abstraction refinement. Since one abstraction usually is not informative enough for challenging planning tasks, we present several ways of creating diverse abstractions. To combine them admissibly, we introduce a new cost partitioning algorithm, which we call saturated cost partitioning. It considers the heuristics sequentially and uses the minimum amount of costs that preserves all heuristic estimates for the current heuristic before passing the remaining costs to subsequent heuristics until all heuristics have been served this way.
In the second part, we show that saturated cost partitioning is strongly influenced by the order in which it considers the heuristics. To find good orders, we present a greedy algorithm for creating an initial order and a hillclimbing search for optimizing a given order. Both algorithms make the resulting heuristics significantly more accurate. However, we obtain the strongest heuristics by maximizing over saturated cost partitioning heuristics computed for multiple orders, especially if we actively search for diverse orders.
The third part provides a theoretical and experimental comparison of saturated cost partitioning and other cost partitioning algorithms. Theoretically, we show that saturated cost partitioning dominates greedy zeroone cost partitioning. The difference between the two algorithms is that saturated cost partitioning opportunistically reuses unconsumed costs for subsequent heuristics. By applying this idea to uniform cost partitioning we obtain an opportunistic variant that dominates the original. We also prove that the maximum over suitable greedy zeroone cost partitioning heuristics dominates the canonical heuristic and show several nondominance results for cost partitioning algorithms. The experimental analysis shows that saturated cost partitioning is the cost partitioning algorithm of choice in all evaluated settings and it even outperforms the previous state of the art in optimal classical planning.

MergeandShrink Abstractions for Classical Planning: Theory, Strategies, and ImplementationPhD thesis, October 2017. Download: (PDF) (slides; PDF) (code) (additional code) (data) (additional data) (scripts)
Classical planning is the problem of finding a sequence of deterministic actions in a state space that lead from an initial state to a state satisfying some goal condition. The dominant approach to optimally solve planning tasks is heuristic search, in particular A* search combined with an admissible heuristic. While there exist many different admissible heuristics, we focus on abstraction heuristics in this thesis, and in particular, on the wellestablished mergeandshrink heuristics.
Our main theoretical contribution is to provide a comprehensive description of the mergeandshrink framework in terms of transformations of transition systems. Unlike previous accounts, our description is fully compositional, i.e. can be understood by understanding each transformation in isolation. In particular, in addition to the namegiving merge and shrink transformations, we also describe pruning and label reduction as such transformations. The latter is based on generalized label reduction, a new theory that removes all of the restrictions of the previous definition of label reduction. We study the four types of transformations in terms of desirable formal properties and explain how these properties transfer to heuristics being admissible and consistent or even perfect. We also describe an optimized implementation of the mergeandshrink framework that substantially improves the efficiency compared to previous implementations.
Furthermore, we investigate the expressive power of mergeandshrink abstractions by analyzing factored mappings, the data structure they use for representing functions. In particular, we show that there exist certain families of functions that can be compactly represented by socalled nonlinear factored mappings but not by linear ones.
On the practical side, we contribute several nonlinear merge strategies to the mergeandshrink toolbox. In particular, we adapt a merge strategy from model checking to planning, provide a framework to enhance existing merge strategies based on symmetries, devise a simple scorebased merge strategy that minimizes the maximum size of transition systems of the mergeandshrink computation, and describe another framework to enhance merge strategies based on an analysis of causal dependencies of the planning task.
In a large experimental study, we show the evolution of the performance of mergeandshrink heuristics on planning benchmarks. Starting with the state of the art before the contributions of this thesis, we subsequently evaluate all of our techniques and show that stateoftheart nonlinear mergeandshrink heuristics improve significantly over the previous state of the art.

New Perspectives on Cost Partitioning for Optimal Classical PlanningPhD thesis, May 2017. Download: (PDF)
Admissible heuristics are the main ingredient when solving classical planning tasks optimally with heuristic search. Higher admissible heuristic values are more accurate, so combining them in a way that dominates their maximum and remains admissible is an important problem.
The thesis makes three contributions in this area. Extensions to cost partitioning (a wellknown heuristic combination framework) allow to produce higher estimates from the same set of heuristics. The new heuristic family called operatorcounting heuristics unifies many existing heuristics and offers a new way to combine them. Another new family of heuristics called potential heuristics allows to cast the problem of finding a good heuristic as an optimization problem.
Both operatorcounting and potential heuristics are closely related to cost partitioning. They offer a new look on cost partitioned heuristics and already sparked research beyond their use as classical planning heuristics.
Master's theses

Compilability between Generalized Representations for Classical PlanningMaster's thesis, December 2022. Download: (PDF) (slides; PDF)
In generalized planning the aim is to solve whole classes of planning tasks instead of single tasks one at a time. Generalized representations provide information or knowledge about such classes to help solving them. This work compares the expressiveness of three generalized representations, generalized potential heuristics, policy sketches and action schema networks, in terms of compilability. We use a notion of equivalence that requires two generalized representations to decompose the tasks of a class into the same subtasks. We present compilations between pairs of equivalent generalized representations and proofs where a compilation is impossible.

Temporal Planning for Droplet Routing on Microfluidic BiochipsMaster's thesis, December 2022. Download: (PDF) (slides; PDF) (sources; ZIP)
A Digital Microfluidic Biochip (DMFB) is a digitally controllable labonachip. Droplets of fluids are moved, merged and mixed on a grid. Routing these droplets efficiently has been tackled by various different approaches. We try to use temporal planning to do droplet routing, inspired by the use of it in quantum circuit compilation. We test a model for droplet routing in both classical and temporal planning and compare both versions. We show that our classical planning model is an efficient method to find droplet routes on DMFBs. Then we extend our model and include spawning, disposing, merging, splitting and mixing of droplets. The results of these extensions show that we are able to find plans for simple experiments. When scaling the problem size to real life experiments our model fails to find plans.

Double Description Method in Cost PartitioningMaster's thesis, November 2022. Download: (PDF) (slides; PDF) (sources; ZIP)
Cost partitioning is a technique used to calculate heuristics in classical optimal planning. It involves solving a linear program. This linear program can be decomposed into a master and pricing problems. In this thesis we combine FourierMotzkin elimination and the double description method in different ways to precompute the generating rays of the pricing problems. We further empirically evaluate these approaches and propose a new method that replaces the FourierMotzkin elimination. Our new method improves the performance of our approaches with respect to runtime and peak memory usage.

Aircraft Maintenance Planning using Constraint ProgrammingMaster's thesis, August 2022.
The increasing number of data nowadays has contributed to new scheduling approaches. Aviation is one of the domains concerned the most, as the aircraft engine implies millions of maintenance events operated by staff worldwide. In this thesis we present a constraint programmingbased algorithm to solve the aircraft maintenance scheduling problem. We want to find the best time to do the maintenance by determining which employee will perform the work and when. Here we report how the scheduling process in aviation can be automatized.

Increasing Horizon Policy Neural Networks for FiniteHorizon MDPsMaster's thesis, April 2022. Download: (PDF) (slides; PDF) (sources; ZIP)
To solve stochastic statespace tasks, the research field of artificial intelligence is mainly used. PROST2014 is state of the art when determining good actions in an MDP environment. In this thesis, we aimed to provide a heuristic by using neural networks to outperform the dominating planning system PROST2014. For this purpose, we introduced two variants of neural networks that allow to estimate the respective Qvalue for a pair of state and action. Since we envisaged the learning method of supervised learning, in addition to the architecture as well as the components of the neural networks, the generation of training data was also one of the main tasks. To determine the most suitable network parameters, we performed a sequential parameter search, from which we expected a local optimum of the model settings. In the end, the PROST2014 planning system could not be surpassed in the total rating evaluation. Nevertheless, in individual domains, we could establish increased final scores on the side of the neural networks. The result shows the potential of this approach and points to eventual adaptations in future work pursuing this procedure furthermore.

Correlation Complexity and Different Notions of WidthMaster's thesis, June 2021. Download: (PDF) (slides; PDF) (sources; ZIP)
In classical planning, there are tasks that are hard and tasks that are easy. We can measure the complexity of a task with the correlation complexity, the improvability width, and the novelty width. In this work, we compare these measures.
We investigate what causes a correlation complexity of at least 2. To do so we translate the state space into a vector space which allows us to make use of linear algebra and convex cones.
Additionally, we introduce the Basel measure, a new measure that is based on potential heuristics and therefore similar to the correlation complexity but also comparable to the novelty width. We show that the Basel measure is a lower bound for the correlation complexity and that the novelty width +1 is an upper bound for the Basel measure.
Furthermore, we compute the Basel measure for some tasks of the International Planning Competitions and show that the translation of a task can increase the Basel measure by removing seemingly irrelevant state variables.

Detecting Unsolvability Based on Parity FunctionsMaster's thesis, April 2021. Download: (PDF) (slides; PDF) (sources; ZIP)
Unsolvability is an important result in classical planning and has seen increased interest in recent years. This thesis explores unsolvability detection by automatically generating parity arguments, a wellknown way of proving unsolvability. The argument requires an invariant measure, whose parity remains constant across all reachable states, while all goal states are of the opposite parity. We express parity arguments using potential functions in the field F_{2} . We develop a set of constraints that describes potential functions with the necessary separating property, and show that the constraints can be represented efficiently for up to twodimensional features. Enhanced with mutex information, an algorithm is formed that tests whether a parity function exists for a given planning task. The existence of such a function proves the task unsolvable. To determine its practical use, we empirically evaluate our approach on a benchmark of unsolvable problems and compare its performance to a state of the art unsolvability planner. We lastly analyze the arguments found by our algorithm to confirm their validity, and understand their expressive power.

A Comparison of Invariant Synthesis MethodsMaster's thesis, October 2020. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
We implemented the invariant synthesis algorithm proposed by Rintanen and experimentally compared it against Helmert’s mutex group synthesis algorithm as implemented in Fast Downward.
The context for the comparison is the translation of propositional STRIPS tasks to FDR tasks, which requires the identification of mutex groups.
Because of its dominating lead in translation speed, combined with few and marginal advantages in performance during search, Helmert’s algorithm is clearly better for most uses. Meanwhile Rintanen’s algorithm is capable of finding invariants other than mutexes, which Helmert’s algorithm per design cannot do.

(Near)optimal policies for Probabilistic IPC 2018 domainsMaster's thesis, June 2020. Download: (PDF) (slides; PDF)
The International Planning Competition (IPC) is a competition of stateoftheart planning systems. The evaluation of these planning systems is done by measuring them with different problems. It focuses on the challenges of AI planning by analyzing classical, probabilistic and temporal planning and by presenting new problems for future research. Some of the probabilistic domains introduced in IPC 2018 are Academic Advising, Chromatic Dice, Cooperative Recon, Manufacturer, Push Your Luck, Redfinned Blueeyes, etc.
This thesis aims to solve (near)optimally two probabilistic IPC 2018 domains, Academic Advising and Chromatic Dice. We use different techniques to solve these two domains. In Academic Advising, we use a relevance analysis to remove irrelevant actions and state variables from the planning task. We then convert the problem from probabilistic to classical planning, which helped us solve it efficiently. In Chromatic Dice, we implement backtracking search to solve the smaller instances optimally. More complex instances are partitioned into several smaller planning tasks, and a nearoptimal policy is derived as a combination of the optimal solutions to the small instances.
The motivation for finding (near)optimal policies is related to the IPC score, which measures the quality of the planners. By providing the optimal upper bound of the domains, we contribute to the stabilization of the IPC score evaluation metric for these domains.

SOGBOFA as heuristic guidance for THTSMaster's thesis, May 2020. Download: (PDF) (slides; PDF) (sources; ZIP)
Most wellknown and traditional online planners for probabilistic planning are in some way based on MonteCarlo Tree Search. SOGBOFA, symbolic online gradientbased optimization for factored action MDPs, offers a new perspective on this: it constructs a function graph encoding the expected reward for a given input state using independence assumptions for states and actions. On this function, they use gradient ascent to perform a symbolic search optimizing the actions for the current state. This unique approach to probabilistic planning has shown very strong results and even more potential. In this thesis, we attempt to integrate the new ideas SOGBOFA presents into the traditionally successful Trialbased Heuristic Tree Search framework. Specifically, we design and evaluate two heuristics based on the aforementioned graph and its Q value estimations, but also the search using gradient ascent. We implement and evaluate these heuristics in the Prost planner, along with a version of the current standalone planner.

Generalization of CycleCovering HeuristicsMaster's thesis, May 2020. Download: (PDF) (slides; PDF) (sources; ZIP)
In this thesis, we consider cyclical dependencies between landmarks for costoptimal planning. Landmarks denote properties that must hold at least once in all plans. However, if the orderings between them induce cyclical dependencies, one of the landmarks in each cycle must be achieved an additional time. We propose the generalized cyclecovering heuristic which considers this in addition to the cost for achieving all landmarks once.
Our research is motivated by recent applications of cyclecovering in the Freecell and logistics domain where it yields nearoptimal results. We carry it over to domainindependent planning using a linear programming approach. The relaxed version of a minimum hitting set problem for the landmarks is enhanced by constraints concerned with cyclical dependencies between them. In theory, this approach surpasses a heuristic that only considers landmarks.
We apply the cyclecovering heuristic in practice where its theoretical dominance is confirmed; Many planning tasks contain cyclical dependencies and considering them affects the heuristic estimates favorably. However, the number of tasks solved using the improved heuristic is virtually unaffected. We still believe that considering this feature of landmarks offers great potential for future work.

Potential Heuristics for Satisficing PlanningMaster's thesis, February 2020. Download: (PDF) (slides; PDF) (sources; ZIP)
Potential heuristics are a class of heuristics used in classical planning to guide a search algorithm towards a goal state. Most of the existing research on potential heuristics is focused on finding heuristics that are admissible, such that they can be used by an algorithm such as A* to arrive at an optimal solution. In this thesis, we focus on the computation of potential heuristics for satisficing planning, where plan optimality is not required and the objective is to find any solution. Specifically, our focus is on the computation of potential heuristics that are descending and deadend avoiding (DDA), since these prop erties guarantee favorable search behavior when used with greedy search algorithms such as hillclimbing. We formally prove that the computation of DDA heuristics is a PSPACEcomplete problem and propose several approximation algorithms. Our evaluation shows that the resulting heuristics are competitive with established approaches such as Pattern Databases in terms of heuristic quality but suffer from several performance bottlenecks.

Planning using Lifted Task RepresentationsMaster's thesis, December 2019. Download: (PDF) (slides; PDF) (sources)
Most automated planners use heuristic search to solve the tasks. Usually, the planners get as input a lifted representation of the task in PDDL, a compact formalism describing the task using a fragment of firstorder logic. The planners then transform this task description into a grounded representation where the task is described in propositional logic. This new grounded format can be exponentially larger than the lifted one, but many planners use this grounded representation because it is easier to implement and reason about.
However, sometimes this transformation between lifted and grounded representations is not tractable. When this is the case, there is not much that planners based on heuristic search can do. Since this transformation is a required preprocess, when this fails, the whole planner fails.
To solve the grounding problem, we introduce new methods to deal with tasks that cannot be grounded. Our work aims to find good ways to perform heuristic search while using a lifted representation of planning problems. We use the pointofview of planning as a database progression problem and borrow solutions from the areas of relational algebra and database theory.
Our theoretical and empirical results are motivating: several instances that were never solved by any planner in the literature are now solved by our new lifted planner. For example, our planner can solve the challenging Organic Synthesis domain using a breadthfirst search, while stateoftheart planners cannot solve more than 60% of the instances. Furthermore, our results offer a new perspective and a deep theoretical study of lifted representations for planning tasks.

Unsolvability Proofs with NonLinear MergeandShrink HeuristicsMaster's thesis, March 2019.
The generation of independently verifiable proofs for the unsolvability of planning tasks using different heuristics, including linear MergeandShrink heuristics, is possible by usage of a proof system framework. Proof generation in the case of nonlinear MergeandShrink heuristic, however, is currently not supported. This is due to the lack of a suitable state set representation formalism that allows to compactly represent states mapped to a certain value in the belonging MergeandShrink representation (MSR). In this thesis, we overcome this shortcoming using Sentential Decision Diagrams (SDDs) as set representations. We describe an algorithm that constructs the desired SDD from the MSR, and show that efficient proof verification is possible with SDDs as representation formalism. Aditionally, we use a proof of concept implementation to analyze the overhead occurred by the proof generation functionality and the runtime of the proof verification.

Evaluation Of PostHoc Optimization Constraints Under Altered Cost FunctionsMaster's thesis, February 2019. Download: (PDF) (slides; PDF) (sources; TGZ)
The operatorcounting framework is a framework in classical planning for heuristics that are based on linear programming. The operatorcounting framework covers several kinds of stateoftheart linear programming heuristics, among them the posthoc optimization heuristic. In this thesis we will use posthoc optimization constraints and evaluate them under altered cost functions instead of the original cost function of the planning task. We show that such costaltered posthoc optimization constraints are also covered by the operatorcounting framework and that it is possible to achieve improved heuristic estimates with them, compared with posthoc optimization constraints under the original cost function. In our experiments we have not been able to achieve improved problem coverage, as we were not able to find a method for generating favorable cost functions that work well in all domains.

NBS applied to PlanningMaster's thesis, February 2019. Download: (PDF) (slides; PDF) (sources; ZIP)
Heuristic forward search is the stateoftheart approach to solve classical planning problems. On the other hand, bidirectional heuristic search has a lot of potential but was never able to deliver on those expectations in practice. Only recently the nearoptimal bidirectional search algorithm (NBS) was introduces by Chen et al. and as the name suggests, NBS expands nearly the optimal number of states to solve any search problem. This is a novel achievement and makes the NBS algorithm a very promising and efficient algorithm in search. With this premise in mind, we raise the question of how applicable NBS is to planning. In this thesis, we inquire this very question by implementing NBS in the state oftheart planner FastDownward and analyse its performance on the benchmark of the latest international planning competition. We additionally implement fractional meetin themiddle and computeWVC to analyse NBS’ performance more thoroughly in regards to the structure of the problem task.
The conducted experiments show that NBS can successfully be applied to planning as it was able to consistently outperform A*. Especially good results were achieved on the domains: blocks, driverlog, floortileopt11strips, getopt14strips, logistics00, and termes opt18strips. Analysing these results, we deduce that the efficiency of forward and backward search depends heavily upon the underlying implicit structure of the transition system which is induced by the problem task. This suggests that bidirectional search is inherently more suited for certain problems. Furthermore, we find that this aptitude for a certain search direction correlates with the domain, thereby providing a powerful analytic tool to a priori derive the effectiveness of certain search approaches.
In conclusion, even without intricate improvements the NBS algorithm is able to compete with A*. It therefore has further potential for future research. Additionally, the underlying transition system of a problem instance is shown to be an important factor which influences the efficiency of certain search approaches. This knowledge could be valuable for devising portfolio planners.

Cost Partitioning Techniques for Multiple Sequence AlignmentMaster's thesis, August 2018. Download: (PDF) (slides; PDF) (sources; ZIP)
Multiple Sequence Alignment (MSA) is the problem of aligning multiple biological sequences in the evoluationary most plausible way. It can be viewed as a shortest path problem through an ndimensional lattice. Because of its large branching factor of 2^n − 1, it has found broad attention in the artificial intelligence community. Finding a globally optimal solution for more than a few sequences requires sophisticated heuristics and bounding techniques in order to solve the problem in acceptable time and within memory limitations. In this thesis, we show how existing heuristics fall into the category of combining certain pattern databases. We combine arbitrary pattern collections that can be used as heuristic estimates and apply cost partitioning techniques from classical planning for MSA. We implement two of those heuristics for MSA and compare their estimates to the existing heuristics.

Using Value Abstraction for Optimal MultiAgent Pathfinding with Increasing Cost Tree SearchMaster's thesis, April 2018. Download: (PDF) (slides; PDF) (sources; ZIP)
Increasing Cost Tree Search is a promising approach to multiagent pathfinding problems, but like all approaches it has to deal with a huge number of possible joint paths, growing exponentially with the number of agents. We explore the possibility of reducing this by introducing a value abstraction to the Multivalued Decision Diagrams used to represent sets of joint paths. To that end we introduce a heat map to heuristically judge how collisionprone agent positions are and present how to use and possible refine abstract positions in order to still find valid paths.

Time Unrolling HeuristicsMaster's thesis, March 2018. Download: (PDF) (slides; PDF) (sources; ZIP)
Estimating cheapest plan costs with the help of network flows is an established technique. Plans and network flows are already very similar, however network flows can differ from plans in the presence of cycles. If a transition system contains cycles, flows might be composed of multiple disconnected parts. This discrepancy can make the cheapest plan estimation worse. One idea to get rid of the cycles works by introducing time steps. For every time step the states of a transition system are copied. Transitions will be changed, so that they connect states only with states of the next time step, which ensures that there are no cycles. It turned out, that by applying this idea to multiple transitions systems, network flows of the individual transition systems can be synchronized via the time steps to get a new kind of heuristic, that will also be discussed in this thesis.

Learning Heuristic Functions Through Supervised LearningMaster's thesis, June 2017. Download: (PDF) (slides; PDF) (sources; ZIP)
Probabilistic planning is a research field that has become popular in the early 1990s. It aims at finding an optimal policy which maximizes the outcome of applying actions to states in an environment that feature unpredictable events. Such environments can consist of a large number of states and actions which make finding an optimal policy intractable using classical methods. Using a heuristic function for a guided search allows for tackling such problems. Designing a domainindependent heuristic function requires complex algorithms which may be expensive when it comes to time and memory consumption.
In this thesis, we are applying the supervised learning techniques for learning two domainindependent heuristic functions. We use three types of gradient descent methods: stochastic, batch and minibatch gradient descent and their improved versions using momen tum, learning decay rate and early stopping. Furthermore, we apply the concept of feature combination in order to better learn the heuristic functions. The learned functions are pro vided to Prost, a domainindependent probabilistic planner, and benchmarked against the winning algorithms of the International Probabilistic Planning Competition held in 2014. The experiments show that learning an offline heuristic improves the overall score of the search for some of the domains used in aforementioned competition.

Merge Strategies for MergeandShrink HeuristicsMaster's thesis, February 2017. Download: (PDF) (slides; PDF) (sources; ZIP)
The mergeandshrink heuristic is a stateoftheart admissible heuristic that is often used for optimal planning. Recent studies showed that the merge strategy is an important factor for the performance of the mergeandshrink algorithm. There are many different merge strategies and improvements for merge strategies described in the literature. One out of these merge strategies is MIASM by Fan et al. MIASM tries to merge transition systems that produce unnecessary states in their product which can be pruned. Another merge strategy is the symmetrybased mergeandshrink framework by Sievers et al. This strategy tries to merge transition systems that cause factored symmetries in their product. This strategy can be combined with other merge strategies and it often improves the performance for many merge strategy. However, the current combination of MIASM with factored symmetries performs worse than MIASM. We implement a different combination of MIASM that uses factored symmetries during the subset search of MIASM. Our experimental evaluation shows that our new combination of MIASM with factored symmetries solves more tasks than the existing MIASM and the previously implemented combination of MIASM with factored symmetries. We also evaluate different combinations of existing merge strategies and find combinations that perform better than their basic version that were not evaluated before.

Pathfinding with TreesMaster's thesis, September 2016. Download: (PDF) (slides; PDF) (sources; ZIP)
Tree Cache is a pathfinding algorithm that selects one vertex as a root and constructs a tree with cheapest paths to all other vertices. A path is found by traversing up the tree from both the start and goal vertices to the root and concatenating the two parts. This is fast, but as all paths constructed this way pass through the root vertex they can be highly suboptimal.
To improve this algorithm, we consider two simple approaches. The first is to construct multiple trees, and save the distance to each root in each vertex. To find a path, the algorithm first selects the root with the lowest total distance. The second approach is to remove redundant vertices, i.e. vertices that are between the root and the lowest common ancestor (LCA) of the start and goal vertices. The performance and space requirements of the resulting algorithm are then compared to the conceptually similar hub labels and differential heuristics.

Combining NoveltyGuided and HeuristicGuided SearchMaster's thesis, July 2016. Download: (PDF) (slides; PDF) (sources; ZIP)
Greedy BestFirst Search (GBFS) is a prominent search algorithm to find solutions for planning tasks. GBFS chooses nodes for further expansion based on a distancetogoal estimator, the heuristic. This makes GBFS highly dependent on the quality of the heuristic. Heuristics often face the problem of producing Uninformed Heuristic Regions (UHRs). GBFS additionally suffers the possibility of simultaneously expanding nodes in multiple UHRs. In this thesis we change the heuristic approach in UHRs. The heuristic was unable to guide the search and so we try to expand novel states to escape the UHRs. The novelty measures how “new” a state is in the search. The result is a combination of heuristic and novelty guided search, which is indeed able to escape UHRs quicker and solve more problems in reasonable time.

Analysing and Combining Static Pruning Techniques for Classical Planning TasksMaster's thesis, May 2016. Download: (PDF) (slides; PDF)
In classical AI planning, the state explosion problem is a reoccurring subject: although the problem descriptions are compact, often a huge number of states needs to be considered. One way to tackle this problem is to use static pruning methods which reduce the number of variables and operators in the problem description before planning.
In this work, we discuss the properties and limitations of three existing static pruning techniques with a focus on satisficing planning. We analyse these pruning techniques and their combinations, and identify synergy effects between them and the domains and problem structures in which they occur. We implement the three methods into an existing propositional planner, and evaluate the performance of different configurations and combinations in a set of experiments on IPC benchmarks. We observe that static pruning techniques can increase the number of solved problems, and that the synergy effects of the combinations also occur on IPC benchmarks, although they do not lead to a major performance increase.

Learning Heuristic Functions in Classical PlanningMaster's thesis, February 2016. Download: (PDF) (slides; PDF) (sources; ZIP)
The goal of classical domainindependent planning is to find a sequence of actions which lead from a given initial state to a goal state that satisfies some goal criteria. Most planning systems use heuristic search algorithms to find such a sequence of actions. A critical part of heuristic search is the heuristic function. In order to find a sequence of actions from an initial state to a goal state efficiently this heuristic function has to guide the search towards the goal. It is difficult to create such an efficient heuristic function. Arfaee et al. show that it is possible to improve a given heuristic function by applying machine learning techniques on a single domain in the context of heuristic search. To achieve this improvement of the heuristic function, they propose a bootstrap learning approach which subsequently improves the heuristic function.
In this thesis we will introduce a technique to learn heuristic functions that can be used in classical domainindependent planning based on the bootstraplearning approach introduced by Arfaee et al. In order to evaluate the performance of the learned heuristic functions, we have implemented a learning algorithm for the Fast Downward planning system. The experiments have shown that a learned heuristic function generally decreases the number of explored states compared to blindsearch. The total time to solve a single problem increases because the heuristic function has to be learned before it can be applied.

Empirical Evaluation of Search Algorithms for Satisficing PlanningMaster's thesis, January 2015. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
Essential for the estimation of the performance of an algorithm in satisficing planning is its ability to solve benchmark problems. Those results can not be compared directly as they originate from different implementations and different machines. We implemented some of the most promising algorithms for greedy bestfirst search, published in the last years, and evaluated them on the same set of benchmarks. All algorithms are either based on randomised search, localised search or a combination of both. Our evaluation proves the potential of those algorithms.

Automatic Selection of Pattern Collections for Domain Independent PlanningMaster's thesis, June 2014. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
Heuristic search with admissible heuristics is the leading approach to costoptimal, domainindependent planning. Pattern database heuristics  a type of abstraction heuristics  are stateoftheart admissible heuristics. Two recent pattern database heuristics are the iPDB heuristic by Haslum et al. and the PhO heuristic by Pommerening et al.
The iPDB procedure performs a hill climbing search in the space of pattern collections and evaluates selected patterns using the canonical heuristic. We apply different techniques to the iPDB procedure, improving its hill climbing algorithm as well as the quality of the resulting heuristic. The second recent heuristic  the PhO heuristic  obtains strong heuristic values through linear programming. We present different techniques to influence and improve on the PhO heuristic.
We evaluate the modified iPDB and PhO heuristics on the IPC benchmark suite and show that these abstraction heuristics can compete with other stateoftheart heuristics in costoptimal, domainindependent planning.

A Case Study on the Search Topology of Greedy BestFirst SearchMaster's thesis, May 2014. Download: (PDF)
Greedy bestfirst search (GBFS) is a prominent search algorithm for satisficing planning  finding good enough solutions to a planning task in reasonable time. GBFS selects the next node to consider based on the most promising node estimated by a heuristic function. However, this behaviour makes GBFS heavily depend on the quality of the heuristic estimator. Inaccurate heuristics can lead GBFS into regions far away from a goal. Additionally, if the heuristic ranks several nodes the same, GBFS has no information on which node it shall follow. Diverse bestfirst search (DBFS) is a new algorithm by Imai and Kishimoto [2011] which has a local search component to emphasis exploitation. To enable exploration, DBFS deploys probabilities to select the next node.
In two problem domains, we analyse GBFS' search behaviour and present theoretical results. We evaluate these results empirically and compare DBFS and GBFS on constructed as well as on provided problem instances.

A General LTL Framework for Describing Control Knowledge in Classical PlanningMaster's thesis, March 2014. Download: (PDF)
Stateoftheart planning systems use a variety of control knowledge in order to enhance the performance of heuristic search. Unfortunately most forms of control knowledge use a specific formalism which makes them hard to combine. There have been several approaches which describe control knowledge in Linear Temporal Logic (LTL). We build upon this work and propose a general framework for encoding control knowledge in LTL formulas. The framework includes a criterion that any LTL formula used in it must fulfill in order to preserve optimal plans when used for pruning the search space; this way the validity of new LTL formulas describing control knowledge can be checked. The framework is implemented on top of the Fast Downward planning system and is tested with a pruning technique called Unnecessary Action Application, which detects if a previously applied action achieved no useful progress.

Efficient Landmark Generation using Relaxed PlanningMaster's thesis, October 2013.
Landmarks are known to be useable for powerful heuristics for informed search. In this thesis, we explain and evaluate a novel algorithm to find ordered landmarks of delete free tasks by intersecting solutions in the relaxation. The proposed algorithm efficiently finds landmarks and natural orders of delete free tasks, such as delete relaxations or Pim compilations.

Action Pruning Through UnderApproximation RefinementMaster's thesis, August 2013. Download: (PDF)
Planning as heuristic search is the prevalent technique to solve planning problems of any kind of domains. Heuristics estimate distances to goal states in order to guide a search through large state spaces. However, this guidance is sometimes moderate, since still a lot of states lie on plateaus of equally prioritized states in the search space topology. Additional techniques that ignore or prefer some actions for solving a problem are successful to support the search in such situations. Nevertheless, some action pruning techniques lead to incomplete searches.
We propose an underapproximation refinement framework for adding actions to underapproximations of planning tasks during a search in order to find a plan. For this framework, we develop a refinement strategy. Starting a search on an initial underapproximation of a planning task, the strategy adds actions determined at states close to a goal, whenever the search does not progress towards a goal, until a plan is found. Key elements of this strategy consider helpful actions and relaxed plans for refinements. We have implemented the underapproximation refinement framework into the greedy best first search algorithm. Our results show considerable speedups for many classical planning problems. Moreover, we are able to plan with fewer actions than standard greedy best first search.

Computation of h+ with Factored PlanningMaster's thesis, July 2013. Download: (PDF)
The main approach for classical planning is heuristic search. Many cost heuristics are based on the delete relaxation. The optimal heuristic of a delete free planning problem is called h^{+}. This thesis explores two new ways to compute h^{+}. Both approaches use factored planning, which decomposes the original planning problem to work on each subproblem separately. The algorithm reuses the subsolutions and combines them to a global solution.
The two algorithms are used to compute a cost heuristic for an A* search. As both approaches compute the optimal heuristic for delete free planning tasks, the algorithms can also be used to find a solution for relaxed planning tasks.

Pebbles in Motion – Polynomial Algorithms for MultiAgent Path Planning ProblemsMaster's thesis, August 2012. Download: (PDF)
MultiAgentPathFinding (MAPF) is a common problem in robotics and memory management. Pebbles in Motion is an implementation of a problem solver for MAPF in polynomial time, based on a work by Daniel Kornhauser from 1984. Recently a lot of research papers have been published on MAPF in the research community of Artificial Intelligence, but the work by Kornhauser seems hardly to be taken into account. We assumed that this might be related to the fact that said paper was more mathematically and hardly describing algorithms intuitively. This work aims at filling this gap, by providing an easy understandable approach of implementation steps for programmers and a new detailed description for researchers in Computer Science.
Bachelor's theses

Deep Reinforcement Learning for Online Planner PortfoliosBachelor's thesis, February 2023. Download: (PDF) (slides; PDF) (sources; ZIP)
In the field of automated planning and scheduling, a planning task is essentially a state space which can be defined rigorously using one of several different formalisms (e.g. STRIPS, SAS+, PDDL etc.). A planning algorithm tries to determine a sequence of actions that lead to a goal state for a given planning task. In recent years, attempts have been made to group certain planners together into so called planner portfolios, to try and leverage their effectiveness on different specific problem classes. In our project, we create an online planner which in contrast to its offline counterparts, makes use of task specific information when allocating a planner to a task. One idea that has recently gained interest, is to apply machine learning methods to planner portfolios.
In previous work such as Delfi (Katz et al., 2018; Sievers et al., 2019a) supervised learning techniques were used, which made it necessary to train multiple networks to be able to attempt multiple, potentially different, planners for a given task. The reason for this being that, if we used the same network, the output would always be the same, as the input to the network would remain unchanged. In this project we make use of techniques from rein forcement learning such as DQNs (Mnih et al., 2013). Using RL approaches such as DQNs, allows us to extend the input to the network to include information on things, such as which planners were previously attempted and for how long. As a result multiple attempts can be made after only having trained a single network.
Unfortunately the results show that current reinforcement learning agents are, amongst other reasons, too sample inefficient to be able to deliver viable results given the size of the currently available data sets.

Optimized Computation of the Additive and FF heuristicsBachelor's thesis, July 2022. Download: (PDF) (slides; PDF) (sources; ZIP)
Planning tasks are important and difficult problems in computer science. A widely used approach is the use of delete relaxation heuristics to which the additive and FF heuristic belong. Those two heuristics use a graph in their calculation, which only has to be constructed once for a planning task but then can be used repeatedly. To solve such a problem efficiently it is important that the calculation of the heuristics are fast. In this thesis the idea to achieve a faster calculation is to combine redundant parts of the graph when building it to reduce the number of edges and therefore speed up the calculation. Here the reduction of the redundancies is done for each action within a planning task individually, but further ideas to simplify over all actions are also discussed.

Bestfirst search with trialbased OpenlistBachelor's thesis, July 2022. Download: (PDF) (slides; PDF) (sources; ZIP)
Monte Carlo search methods are widely known, mostly for their success in game domains, although they are also applied to many nongame domains. In previous work done by Schulte and Keller, it was established that bestfirst searches could adapt to the action selection functionality which make Monte Carlo methods so formidable. In practice however, the trialbased best first search, without exploration, was shown to be slightly slower than its explicit open list counterpart. In this thesis we examine the nontrial and trialbased searches and how they can address the exploitation exploration dilemma. Lastly, we will see how trialbased BFS can rectify a slower search by allowing occasional random action selection, by comparing it to regular open list searches in a line of experiments.

Encoding Diverse Sudoku Variants as SAT ProblemsBachelor's thesis, July 2022. Download: (PDF) (slides; PDF) (sources; ZIP)
Sudoku has become one of the world’s most popular logic puzzles, arousing interest in the general public and the science community. Although the rules of Sudoku may seem simple, they allow for nearly countless puzzle instances, some of which are very hard to solve. SATsolvers have proven to be a suitable option to solve Sudokus automatically. However, they demand the puzzles to be encoded as logical formulae in Conjunctive Normal Form. In earlier work, such encodings have been successfully demonstrated for original Sudoku Puzzles. In this thesis, we present encodings for rather unconventional Sudoku Variants, developed by the puzzle community to create even more challenging solving experiences. Furthermore, we demonstrate how PseudoBoolean Constraints can be utilized to encode Sudoku Variants that follow rules involving sums. To implement an encoding of PseudoBoolean Constraints, we use Binary Decision Diagrams and Adder Networks and study how they compare to each other.

Generation of Domain Abstractions using CounterexampleGuided Abstraction RefinementBachelor's thesis, June 2022. Download: (PDF) (slides; PDF) (sources; ZIP)
In optimal classical planning, informed search algorithms like A* need admissible heuristics to find optimal solutions. Counterexampleguided abstraction refinement (CEGAR) is a method used to generate abstractions that yield suitable abstraction heuristics iteratively. In this thesis, we propose a class of CEGAR algorithms for the generation of domain abstractions, which are a class of abstractions that rank in between projections and Cartesian abstractions regarding the grade of refinement they allow. As no known algorithm constructs domain abstractions, we show that our algorithm is competitive with CEGAR algorithms that generate one projection or Cartesian abstraction.

SinglePlayer Chess as a Planning ProblemBachelor's thesis, June 2022. Download: (PDF) (slides; PDF) (sources; ZIP)
This thesis will look at SinglePlayer Chess as a planning domain using two approaches: one where we look at how we can encode the SinglePlayer Chess problem as a domainindependent (generalpurpose AI) approach and one where we encode the problem as a domainspecific solver. Lastly, we will compare the two approaches by doing some experiments and comparing the results of the two approaches. Both the domainindependent implementation and the domainspecific implementation differ from traditional chess engines because the task of the agent is not to find the best move for a given position and colour, but the agent’s task is to check if a given chess problem has a solution or not. If the agent can find a solution, the given chess puzzle is valid. The results of both approaches were measured in experiments, and we found out that the domainindependent implementation is too slow and that the domainspecific implementation, on the other hand, can solve the given puzzles reliably, but it has a memory bottleneck rooted in the search method that was used.

Monte Carlo Tree Search for CarcassonneBachelor's thesis, June 2022. Download: (PDF) (slides; PDF) (sources; ZIP)
Carcassonne is a tilebased board game with a large state space and a high branching factor and therefore poses a challenge to artificial intelligence. In the past, Monte Carlo Tree Search (MCTS), a search algorithm for sequential decisionmaking processes, has been shown to find good solutions in large state spaces. MCTS works by iteratively building a game tree according to a tree policy. The profitability of paths within that tree is evaluated using a default policy, which influences in what directions the game tree is expanded. The functionality of these two policies, as well as other factors, can be implemented in many different ways. In consequence, many different variants of MCTS exist. In this thesis, we applied MCTS to the domain of twoplayer Carcassonne and evaluated different variants in regard to their performance and runtime. We found significant differences in performance for various variable aspects of MCTS and could thereby evaluate a configuration which performs best on the domain of Carcassonne. This variant consistently outperformed an average human player with a feasible runtime.

Optimality Certificates for Classical PlanningBachelor's thesis, June 2022. Download: (PDF) (slides; PDF) (sources; ZIP)
In general, it is important to verify software as it is prone to error. This also holds for solving tasks in classical planning. So far, plans in general as well as the fact that there is no plan for a given planning task can be proven and independently verified. However, no such proof for the optimality of a solution of a task exists. Our aim is to introduce two methods with which optimality can be proven and independently verified. We first reduce unit cost tasks to unsolvable tasks, which enables us to make use of the already existing certificates for unsolvability. In a second approach, we propose a proof system for optimality, which enables us to infer that the determined cost of a task is optimal. This permits the direct generation of optimality certificates.

Posthoc Optimization for the Sliding Tile PuzzleBachelor's thesis, October 2021. Download: (PDF) (slides; PDF) (sources; ZIP)
Pattern databases are one of the most powerful heuristics in classical planning. They evaluate the perfect cost for a simplified subproblem. The posthoc optimization heuristic is a technique on how to optimally combine a set of pattern databases. In this thesis, we will adapt the posthoc optimization heuristic for the sliding tile puzzle. The sliding tile puzzle serves as a benchmark to compare the posthoc optimization heuristic to already established methods, which also deal with the combining of pattern databases. We will then show how the posthoc optimization heuristic is an improvement over the already established methods.

Evaluating the Cyclic Landmark Heuristic with a Logisticsspecific Landmark GeneratorBachelor's thesis, September 2021. Download: (PDF) (slides; PDF) (sources; ZIP)
In this thesis, we generate landmarks for a logisticsspecific task. Landmarks are actions that need to occur at least once in every plan. A landmark graph denotes a structure with landmarks and their edges called orderings. If there are cycles in a landmark graph, one of those landmarks needs to be achieved at least twice for every cycle. The generation of the logisticsspecific landmarks and their orderings calculate the cyclic landmark heuristic. The task is to pick up on related work, the evaluation of the cyclic landmark heuristic. We compare the generation of landmark graphs from a domainindependent landmark generator to a domainspecific landmark generator, the latter being the focus. We aim to bridge the gap between domainspecific and domainindependent landmark generators. In this thesis, we compare one domainspecific approach for the logistics domain with results from a domain independent landmark generator. We devise a unit to preprocess data for other domain specific tasks as well. We will show that specificity is better suited than independence.

Optimierungsanfragen für Town Building Spiele mithilfe von LP & IPSolverBachelor's thesis, August 2021. Download: (PDF) (slides; PDF) (sources; ZIP)
Lineare Programmierung ist eine mathematische Modellierungstechnik, bei der eine lineare Funktion, unter der Berücksichtigung verschiedenen Beschränkungen, maximiert oder minimiert werden soll. Diese Technik ist besonders nützlich, falls Entscheidungen für Optimierungsprobleme getroffen werden sollen. Ziel dieser Arbeit war es ein Tool für das Spiel Factory Town zu entwickeln, mithilfe man Optimierungsanfragen bearbeiten kann. Dabei ist es möglich wahlweise zwischen diversen Fragestellungen zu wählen und anhand von LP\ IPSolvern diese zu beantworten. Zudem wurden die mathematischen Formulierungen, sowie die Unterschiede beider Methoden angegangen. Schlussendlich unterstrichen die generierten Resultate, dass LP Lösungen mindestens genauso gut oder sogar besser seien als die Lösungen eines IP.

Factored Mappings as Knowledge Compilation for Symbolic SearchBachelor's thesis, June 2021. Download: (PDF) (slides; PDF)
Symbolic search is an important approach to classical planning. Symbolic search uses search algorithms that process sets of states at a time. For this we need states to be represented by a compact data structure called knowledge compilations. Mergeandshrink representations come a different field of planning, where they have been used to derive heuristic functions for statespace search. More generally they represent functions that map variable assignments to a set of values, as such we can regard them as a data structure we will call Factored Mappings. In this thesis, we will investigate Factored Mappings (FMs) as a knowledge compilation language with the hope of using them for symbolic search. We will analyse the necessary transformations and queries for FMs, by defining the needed operations and a canonical representation of FMs, and showing that they run in polynomial time. We will then show that it is possible to use Factored Mappings as a knowledge compilation for symbolic search by defining a symbolic search algorithm for a finitedomain plannings task that works with FMs.

Modelling Git Operations as Planning ProblemsBachelor's thesis, January 2021. Download: (PDF) (slides; PDF) (sources; ZIP)
Version control systems use a graph data structure to track revisions of files. Those graphs are mutated with various commands by the respective version control system. The goal of this thesis is to formally define a model of a subset of Git commands which mutate the revision graph, and to model those mutations as a planning task in the Planning Domain Definition Language. Multiple ways to model those graphs will be explored and those models will be compared by testing them using a set of planners.

Compressed Pattern Databases for Classical PlanningBachelor's thesis, December 2020. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
Pattern Databases are admissible abstraction heuristics for classical planning. In this thesis we are introducing the Boosting processes, which consists of enlarging the pattern of a Pattern Database P, calculating a more informed Pattern Database P' and then mincompress P' to the size of P resulting in a compressed and still admissible Pattern Database P''. We design and implement two boosting algorithms, Hillclimbing and Randomwalk.
We combine pattern database heuristics using five different cost partitioning methods. The experiments compare computing cost partitionings over regular and boosted pattern databases. The experiments, performed on IPC (optimal track) tasks, show promising results which increased the coverage (number of solved tasks) by 9 for canonical cost partitioning using our Randomwalk boosting variant.

Mutex Based Potential HeuristicsBachelor's thesis, November 2020. Download: (PDF) (slides; PDF) (sources; ZIP)
One dimensional potential heuristics assign a numerical value, the potential, to each fact of a classical planning problem. The heuristic value of a state is the sum over the poten tials belonging to the facts contained in the state. Fišer et al. (2020) recently proposed to strengthen potential heuristics utilizing mutexes and disambiguations. In this thesis, we embed the same enhancements in the planning system Fast Downward. The experi mental evaluation shows that the strengthened potential heuristics are a refinement, but too computationally expensive to solve more problems than the nonstrengthened potential heuristics.
The potentials are obtained with a Linear Program. Fišer et al. (2020) introduced an additional constraint on the initial state and we propose additional constraints on random states. The additional constraints improve the amount of solved problems by up to 5%.

Exploring The Prioritized Incremental HeuristicBachelor's thesis, November 2020. Download: (PDF) (slides; PDF) (sources; ZIP)
This thesis discusses the PINCH heuristic, a specific implementation of the additive heuristic. PINCH intends to combine the strengths of existing implementations of the additive heuristic. The goal of this thesis is to really dig into the PINCH heuristic. I want to provide the most accessible resource for understanding PINCH and I want to analyze the performance of PINCH by comparing it to the algorithm on which it is based, Generalized Dijkstra.

Bounded Suboptimal Search for Classical PlanningBachelor's thesis, October 2020. Download: (PDF) (slides; PDF) (sources; ZIP)
Suboptimal search algorithms can offer attractive benefits compared to optimal search, namely increased coverage of larger search problems and quicker search times. Improving on such algorithms, such as reducing costs further towards optimal solutions and reducing the number of node expansions, is therefore a compelling area for further research. This paper explores the utility and scalability of recently developed priority functions, XDP, XUP, and PWXDP, and the Improved Optimistic Search algorithm, compared to Weighted A*, in the Fast Downward planner. Analyses focus on the cost, total time, coverage, and node expansion parameters, with experimental evidence suggesting preferable performance if strict optimality is not desired. The implementation of priorityb functions in eager bestfirst search showed marked improvements compared to A* search on coverage, total time, and number of expansions, without significant cost penalties. Following previous suboptimal search research, experimental evidence even seems to indicate that these cost penalties do not reach the designated bound, even in larger search spaces.

Finding Small Counterexamples of Expected Planner Behavior with HillClimbingBachelor's thesis, October 2020. Download: (PDF) (sources; TAR.GZ)
In the Automated Planning field, algorithms and systems are developed for exploring state spaces and ultimately finding an action sequence leading from a task’s initial state to its goal. Such planning systems may sometimes show unexpected behavior, caused by a planning task or a bug in the planner itself. Generally speaking, finding the source of a bug tends to be easier when the cause can be isolated or simplified. In this thesis, we tackle this problem by making PDDL and SAS+ tasks smaller while ensuring they still invoke a certain characteristic when executed with a planner. We implement a system that successively removes elements, such as objects, from a task and checks whether the transformed task still fails on the planner. Elements are removed in a syntactically consistent way, however, no semantic integrity is enforced. Our system’s design is centered around the Fast Downward Planning System, as we reuse some of its translator modules and all test runs are performed with Fast Downward. At the core of our system, firstchoice hillclimbing is used for optimization. Our “minimizer” takes (1) a failing planner execution command, (2) a description of the failing characteristic and (3) the type of element to be deleted as arguments. We evaluate our system’s functionality on the basis of three usecases. In our most successful test runs, (1) a SAS+ task with initially 1536 operators and 184 variables is reduced to 2 operators and 2 variables and (2)a PDDL task with initially 46 actions, 62 objects and 29 predicate symbols is reduced to 2 actions, 6 objects and 4 predicates.

Implementing and Evaluating Successor Generators in the Fast Downward Planning SystemBachelor's thesis, September 2020. Download: (PDF) (slides; PDF) (sources; ZIP)
Fast Downward is a classical planning system based on heuristic search. Its successor generator is an efficient and intelligent tool to process state spaces and generate their successor states. In this thesis we implement different successor generators in the Fast Downward planning system and compare them against each other. Apart from the given fast downward successor generator we implement four other successor generators: a naive successor generator, one based on the marking of delete relaxed heuristics, one based on the PSVN planning system and one based on watched literals as used in modern SAT solvers. These successor generators are tested in a variety of different planning benchmarks to see how well they compete against each other. We verified that there is a tradeoff between precomputation and faster successor generation and showed that all of the implemented successor generators have a use case and it is advisable to switch to a successor generator that fits the style of the planning task.

Extending SymPA with Unsolvability CertificatesBachelor's thesis, September 2020. Download: (PDF) (slides; PDF) (sources; ZIP)
Verifying whether a planning algorithm came to the correct result for a given planning task is easy if a plan is emitted which solves the problem. But if a task is unsolvable most planners just state this fact without any explanation or even proof. In this thesis we present extended versions of the symbolic search algorithms SymPA and symbolic bidirectional uniformcost search which, if a given planning task is unsolvable, provide certificates which prove unsolvability. We also discuss a concrete implementation of this version of SymPA.

Concept Languages as Expert Input for Generalized PlanningBachelor's thesis, June 2020. Download: (PDF) (slides; PDF) (sources)
Classical planning is an attractive approach to solving problems because of its generality and its relative ease of use. Domainspecific algorithms are appealing because of their performance, but require a lot of resources to be implemented. In this thesis we evaluate concepts languages as a possible input language for expert domain knowledge into a planning system. We also explore mixed integer programming as a way to use this knowledge to improve search efficiency and to help the user find and refine useful domain knowledge.

A Formal Verification of Strong Stubborn Set Based PruningBachelor's thesis, May 2020. Download: (PDF) (slides; PDF) (sources; ZIP)
Classical Planning is a branch of artificial intelligence that studies single agent, static, deterministic, fully observable, discrete search problems. A common challenge in this field is the explosion of states to be considered when searching for the goal. One technique that has been developed to mitigate this is Strong Stubborn Set based pruning, where on each state expansion, the considered successors are restricted to Strong Stubborn Sets, which exploit the properties of independent operators to cut down the tree or graph search. We adopt the definitions of the theory of Strong Stubborn Sets from the SAS+ setting to transition systems and validate a central theorem about the correctness of Strong Stubborn Set based pruning for transition systems in the interactive theorem prover Isabelle/HOL.

Change Detection in THTSBachelor's thesis, September 2019. Download: (PDF) (sources; TAR.GZ)
Ein wichtiges Feld in der Wissenschaft der künstliche Intelligenz sind Planungsprobleme. Man hat das Ziel, eine künstliche intelligente Maschine zu bauen, die mit so vielen ver schiedenen Probleme umgehen und zuverlässig lösen kann, indem sie ein optimaler Plan herstellt.
Der Trialbased Heuristic Tree Search(THTS) ist ein mächtiges Werkzeug um MultiArmed Banditähnliche Probleme, Marcow Decsision Processe mit verändernden Rewards, zu lösen. Beim momentanen THTS können explorierte gefundene gute Rewards auf Grund von der grossen Anzahl der Rewards nicht beachtet werden. Ebenso können beim explorieren schlech te Rewards, gute Knoten im Suchbaum, verschlechtern. Diese Arbeit führt eine Methodik ein, die von der stückweise stationären MABs Problematik stammt, um den THTS weiter zu optimieren.

Refinement Strategies for CounterexampleGuided Cartesian Abstraction RefinementBachelor's thesis, July 2019. Download: (PDF) (slides; PDF) (sources; ZIP)
Abstractions are a simple yet powerful method of creating a heuristic to solve classical planning problems optimally. In this thesis we make use of Cartesian abstractions generated with CounterexampleGuided Abstraction Refinement (CEGAR). This method refines abstractions incrementally by finding flaws and then resolving them until the abstraction is sufficiently evolved. The goal of this thesis is to implement and evaluate algorithms which select solutions of such flaws, in a way which results in the best abstraction (that is, the abstraction which causes the problem to then be solved most efficiently by the planner). We measure the performance of a refinement strategy by running the Fast Downward planner on a problem and measuring how long it takes to generate the abstraction, as well as how many expansions the planner requires to find a goal using the abstraction as a heuristic. We use a suite of various benchmark problems for evaluation, and we perform this experiment for a single abstraction and on abstractions for multiple subtasks. Finally, we attempt to predict which refinement strategy should be used based on parameters of the task, potentially allowing the planner to automatically select the best strategy at runtime.

Heuristic Planning with Single Action Goal ExpansionBachelor's thesis, June 2019. Download: (PDF) (slides; PDF) (sources; ZIP)
Heuristic search is a powerful paradigm in classical planning. The information generated by heuristic functions to guide the search towards a goal is a key component of many modern search algorithms. The paper “Using Backwards Generated Goals for Heuristic Planning” by Alcázar et al. proposes a way to make additional use of this information. They take the last actions of a relaxed plan as a basis to generate intermediate goals with a known path to the original goal. A plan is found when the forward search reaches an intermediate goal.
The premise of this thesis is to modify their approach by focusing on a single sequence of intermediate goals. The aim is to improve efficiency while preserving the benefits of backwards goal expansion. We propose different variations of our approach by introducing multiple ways to make decisions concerning the construction of intermediate goals. We evaluate these variations by comparing their performance and illustrate the challenges posed by this approach.

Computing Abstract Plans for CounterexampleGuided Cartesian Abstraction RefinementBachelor's thesis, May 2019. Download: (PDF) (slides; PDF) (sources; ZIP)
Counterexampleguided abstraction refinement (CEGAR) is a way to incrementally compute abstractions of transition systems. It starts with a coarse abstraction and then iteratively finds an abstract plan, checks where the plan fails in the concrete transition system and refines the abstraction such that the same failure cannot happen in subsequent iterations. As the abstraction grows in size, finding a solution for the abstract system becomes more and more costly. Because the abstraction grows incrementally, however, it is possible to maintain heuristic information about the abstract state space, allowing the use of informed search algorithms like A*. As the quality of the heuristic is crucial to the performance of informed search, the method for maintaining the heuristic has a significant impact on the performance of the abstraction refinement as a whole. In this thesis, we investigate different methods for maintaining the value of the perfect heuristic h* at all times and evaluate their performance.

Pattern Selection using Counterexampleguided Abstraction RefinementBachelor's thesis, July 2018. Download: (PDF) (slides; PDF) (sources; ZIP)
Pattern Databases are a powerful class of abstraction heuristics which provide admissible path cost estimates by computing exact solution costs for all states of a smaller task. Said task is obtained by abstracting away variables of the original problem. Abstractions with few variables offer weak estimates, while introduction of additional variables is guaranteed to at least double the amount of memory needed for the pattern database. In this thesis, we present a class of algorithms based on counterexampleguided abstraction refinement (CEGAR), which exploit additivity relations of patterns to produce pattern collections from which we can derive heuristics that are both informative and computationally tractable. We show that our algorithms are competitive with already existing pattern generators by comparing their performance on a variety of planning tasks.

Abstraction Heuristics for Rubik’s CubeBachelor's thesis, June 2018. Download: (PDF) (slides; PDF) (sources; ZIP)
We consider the problem of Rubik’s Cube to evaluate modern abstraction heuristics. In order to find feasible abstractions of the enormous state space spanned by Rubik’s Cube, we apply projection in the form of pattern databases, Cartesian abstraction by doing counterexample guided abstraction refinement as well as mergeandshrink strategies. While previous publications on Cartesian abstractions have not covered applicability for planning tasks with conditional effects, we introduce factorized effect tasks and show that Cartesian abstraction can be applied to them. In order to evaluate the performance of the chosen heuristics, we run experiments on different problem instances of Rubik’s Cube. We compare them by the initial hvalue found for all problems and analyze the number of expanded states up to the last flayer. These criteria provide insights about the informativeness of the considered heuristics. Cartesian Abstraction yields perfect heuristic values for problem instances close to the goal, however it is outperformed by pattern databases for more complex instances. Even though mergeandshrink is the most general abstraction among the considered, it does not show better performance than the others.

Metareasoning for Deliberation Time Distribution in the Prost PlannerBachelor's thesis, December 2017. Download: (PDF) (slides; PDF) (sources; ZIP)
Probabilistic planning expands on classical planning by tying probabilities to the effects of actions. Due to the exponential size of the states, probabilistic planners have to come up with a strong policy in a very limited time. One approach to optimising the policy that can be found in the available time is called metareasoning, a technique aiming to allocate more deliberation time to steps where more time to plan results in an improvement of the policy and less deliberation time to steps where an improvement of the policy with more time to plan is unlikely.
This thesis aims to adapt a recent proposal of a formal metareasoning procedure from Lin. et al. for the search algorithm BRTDP to work with the UCT algorithm in the Prost planner and compare its viability to the current standard and a number of less informed time management methods in order to find a potential improvement to the current uniform deliberation time distribution.

Enhancing Prost with Abstraction of StateAction PairsBachelor's thesis, December 2017. Download: (PDF) (slides; PDF) (sources; ZIP)
A planner tries to produce a policy that leads to a desired goal given the available range of actions and an initial state. A traditional approach for an algorithm is to use abstraction. In this thesis we implement the algorithm described in the ASAPUCT paper: Abstraction of StateAction Pairs in UCT by Ankit Anand, Aditya Grover, Mausam and Parag Singla.
The algorithm combines state and stateaction abstraction with a UCTalgorithm. We come to the conclusion that the algorithm needs to be improved because the abstraction of actionstate often cannot detect a similarity that a reasonable action abstraction could find.

Combining NoveltyGuided and Bounded Suboptimal SearchBachelor's thesis, July 2017. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
The notion of adding a form of exploration to guide a search has been proven to be an effective method of combating heuristical plateaus and improving the performance of greedy bestfirst search. The goal of this thesis is to take the same approach and introduce exploration in a bounded suboptimal search problem. Explicit estimation search (EES), established by Thayer and Ruml, consults potentially inadmissible information to determine the search order. Admissible heuristics are then used to guarantee the cost bound. In this work we replace the distancetogo estimator used in EES with an approach based on the concept of novelty.

Solving DeleteRelaxed Planning Tasks by Using Cut SetsBachelor's thesis, June 2017. Download: (PDF) (slides; PDF) (sources; ZIP)
Classical domainindependent planning is about finding a sequence of actions which lead from an initial state to a goal state. A popular approach for solving planning problems efficiently is to utilize heuristic functions. A possible heuristic function is the perfect heuristic of a delete relaxed planning problem denoted as h+. Delete relaxation simplifies the planning problem thus making it easier to find a perfect heuristic. However computing h+ is still NPhard problem.
In this thesis we discuss a promising looking approach to compute h+ in practice. Inspired by the paper from Gnad, Hoffmann and Domshlak about starshaped planning problems, we implemented the FlowCut algorithm. The basic idea behind flowcut to divide a problem that is unsolvable in practice, into smaller sub problems that can be solved. We further tested the flowcut algorithm on the domains provided by the International Planning Competition benchmarks, resulting in the following conclusion: Using a divide and conquer approach can successfully be used to solve classical planning problems, however it is not trivial to design such an algorithm to be more efficient than stateoftheart search algorithm.

Landmarkbased Meta BestFirst SearchBachelor's thesis, June 2017. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
This thesis deals with the algorithm presented in the paper "Landmarkbased Meta BestFirst Search Algorithm: First Parallelization Attempt and Evaluation" by Simon Vernhes, Guillaume Infantes and Vincent Vidal. Their idea was to reconsider the approach to landmarks as a tool in automated planning, but in a markedly different way than previous work had done. Their result is a metasearch algorithm which explores landmark orderings to find a series of subproblems that reliably lead to an effective solution. Any complete planner may be used to solve the subproblems. While the referenced paper also deals with an attempt to effectively parallelize the Landmarkbased Meta BestFirst Search Algorithm, this thesis is concerned mainly with the sequential implementation and evaluation of the algorithm in the Fast Downward planning system.

DepthBound Heuristics and IterativeDeepening Search Algorithms in Classical PlanningBachelor's thesis, June 2017. Download: (PDF) (slides; PDF) (sources; ZIP)
Heuristics play an important role in classical planning. Using heuristics during state space search often reduces the time required to find a solution, but constructing heuristics and using them to calculate heuristic values takes time, reducing this benefit. Constructing heuristics and calculating heuristic values as quickly as possible is very important to the effectiveness of a heuristic. In this thesis we introduce methods to bound the construction of mergeandshrink to reduce its construction time and increase its accuracy for small problems and to bound the heuris tic calculation of landmark cut to reduce heuristic value calculation time. To evaluate the performance of these depthbound heuristics we have implemented them in the Fast Down ward planning system together with three iterativedeepening heuristic search algorithms: iterativedeepening A* search, a new breadthfirst iterativedeepening version of A* search and iterativedeepening breadthfirst heuristic search.

Diversifying Greedy BestFirst Search by Clustering StatesBachelor's thesis, March 2017. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
Greedy bestfirst search has proven to be a very efficient approach to satisficing planning but can potentially lose some of its effectiveness due to the used heuristic function misleading it to a local minimum or plateau. This is where exploration with additional open lists comes in, to assist greedy bestfirst search with solving satisficing planning tasks more effectively. Building on the idea of exploration by clustering similar states together as described by Xie et al. [2014], where states are clustered according to heuristic values, we propose in this paper to instead cluster states based on the Hamming distance of the binary representation of states [Hamming, 1950]. The resulting open list maintains k buckets and inserts each given state into the bucket with the smallest average hamming distance between the already clustered states and the new state. Additionally, our open list is capable of reclustering all states periodically with the use of the kmeans algorithm. We were able to achieve promising results concerning the amount of expansions necessary to reach a goal state, despite not achieving a higher coverage than fully random exploration due to slow performance. This was caused by the amount of calculations required to identify the most fitting cluster when inserting a new state.

Online Knowledge Enhancements for MonteCarlo Tree Search in Probabilistic PlanningBachelor's thesis, February 2017. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
Monte Carlo Tree Search Algorithms are an efficient method of solving probabilistic planning tasks that are modeled by Markov Decision Problems. MCTS uses two policies, a tree policy for iterating through the known part of the decission tree and a default policy to simulate the actions and their reward after leaving the tree. MCTS algorithms have been applied with great success to computer Go. To make the two policies fast many enhancements based on online knowledge have been developed. The goal of All Moves as First enhancements is to improve the quality of a reward estimate in the tree policy. In the context of this thesis the, in the field of computer Go very efficient, αAMAF, CutoffAMAF as well as Rapid Action Value Estimation enhancements are implemented in the probabilistic planner PROST. To obtain a better default policy, Move Average Sampling is implemented into PROST and benchmarked against it’s current default policies.

PartitionBased Pruning for Classical Planning RevisitedBachelor's thesis, January 2017. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
In classical planning the objective is to find a sequence of applicable actions that lead from the initial state to a goal state. In many cases the given problem can be of enormous size. To deal with these cases, a prominent method is to use heuristic search, which uses a heuristic function to evaluate states and can focus on the most promising ones. In addition to applying heuristics, the search algorithm can apply additional pruning techniques that exclude applicable actions in a state because applying them at a later point in the path would result in a path consisting of the same actions but in a different order. The question remains as to how these actions can be selected without generating too much additional work to still be useful for the overall search. In this thesis we implement and evaluate the partitionbased path pruning method, proposed by Nissim et al. [1], which tries to decompose the set of all actions into partitions. Based on this decomposition, actions can be pruned with very little additional information. The partitionbased pruning method guarantees with some alterations to the A* search algorithm to preserve it’s optimality. The evaluation confirms that in several standard planning domains, the pruning method can reduce the size of the explored state space.

UnderApproximation Refinement for Timed AutomataBachelor's thesis, January 2017. Download: (PDF) (slides; PDF) (sources; ZIP)
Validating realtime systems is an important and complex task which becomes exponentially harder with increasing sizes of systems. Therefore finding an automated approach to check realtime systems for possible errors is crucial. The behaviour of such realtime systems can be modelled with timed automata. This thesis adapts and implements the underapproximation refinement algorithm developed for search based planners proposed by Heusner et al. to find error states in timed automata via the directed model checking approach. The evaluation compares the algorithm to already existing search methods and shows that a basic underapproximation refinement algorithm yields a competitive search method for directed model checking which is both fast and memory efficient. Additionally we illustrate that with the introduction of some minor alterations the proposed under approximation refinement algorithm can be further improved.

Berechnung von PotentialHeuristiken basierend auf PatternDatenbankenBachelor's thesis, December 2016. Download: (PDF) (slides; PDF) (sources; ZIP)
In dieser Arbeit wird versucht eine Heuristik zu lernen. Damit eine Heuristik erlernbar ist, muss sie über Parameter verfügen, die die Heuristik bestimmen. Eine solche Möglichkeit bieten PotentialHeuristiken und ihre Parameter werden Potentiale genannt. PatternDatabases können mit vergleichsweise wenig Aufwand Eigenschaften eines Zustandsraumes erkennen und können somit eingesetzt werden als Grundlage um Potentiale zu lernen. Diese Arbeit untersucht zwei verschiedene Ansätze zum Erlernen der Potentiale aufgrund der Information aus PatternDatabases. In Experimenten werden die beiden Ansätze genauer untersucht und schliesslich mit der FFHeuristik verglichen.

A Formalism for Build Order Search in StarCraft Brood WarBachelor's thesis, December 2016. Download: (PDF) (slides; PDF) (sources; ZIP)
We consider realtime strategy (RTS) games which have temporal and numerical aspects and pose challenges which have to be solved within limited search time. These games are interesting for AI research because they are more complex than board games. Current AI agents cannot consistently defeat average human players, while even the best players make mistakes we think an AI could avoid. In this thesis, we will focus on StarCraft Brood War. We will introduce a formal definition of the model Churchill and Buro proposed for StarCraft. This allows us to focus on Build Order optimization only. We have implemented a base version of the algorithm Churchill and Buro used for their agent. Using the implementation we are able to find solutions for Build Order Problems in StarCraft Brood War.

Symbolische Zustandsraumsuche mit Sentential Decision DiagramsBachelor's thesis, September 2016. Download: (PDF) (slides; PDF) (sources; ZIP)
Auf dem Gebiet der Handlungsplanung stellt die symbolische Suche eine der erfolgversprechendsten angewandten Techniken dar. Um eine symbolische Suche auf endlichen Zustandsräumen zu implementieren bedarf es einer geeigneten Datenstruktur für logische Formeln. Diese Arbeit erprobt die Nutzung von Sentential Decision Diagrams (SDDs) anstelle der gängigen Binary Decision Diagrams (BDDs) zu diesem Zweck. SDDs sind eine Generalisierung von BDDs. Es wird empirisch getestet wie eine Implementierung der symbolischen Suche mit SDDs im FastDownwardPlaner sich mit verschiedenen vtrees unterscheidet. Insbesondere wird die Performance von balancierten vtrees, mit welchen die Stärken von SDDs oft gut zur Geltung kommen, mit rechtsseitig linearen vtrees verglichen, bei welchen sich SDDs wie BDDs verhalten.

Gibt es Sudokus mit nur 16 Vorgaben?Bachelor's thesis, September 2016. Download: (PDF) (slides; PDF) (sources; ZIP)
Die Frage ob es gültige Sudokus  d.h. Sudokus mit nur einer Lösung  gibt, die nur 16 Vorgaben haben, konnte im Dezember 2011 mithilfe einer erschöpfenden BruteForceMethode von McGuire et al. verneint werden. Die Schwierigkeit dieser Aufgabe liegt in dem ausufernden Suchraum des Problems und der dadurch entstehenden Erforderlichkeit einer effizienten Beweisidee sowie schnellerer Algorithmen. In dieser Arbeit wird die Beweismethode von McGuire et al. bestätigt werden und für 2^{2} × 2^{2} und 3^{2} × 3^{2} Sudokus in C++ implementiert.

Komprimierte Pfaddatenbanken durch Lauflängenkodierung von optimal permutierten firstmove MatrizenBachelor's thesis, June 2016. Download: (PDF) (slides; PDF) (sources; ZIP)
Das Finden eines kürzesten Pfades zwischen zwei Punkten ist ein fundamentales Problem in der Graphentheorie. In der Praxis ist es oft wichtig, den Ressourcenverbrauch für das Ermitteln eines solchen Pfades minimal zu halten, was mithilfe einer komprimierten Pfaddatenbank erreicht werden kann. Im Rahmen dieser Arbeit bestimmen wir drei Verfahren, mit denen eine Pfaddatenbank möglichst platzsparend aufgestellt werden kann, und evaluieren die Effektivität dieser Verfahren anhand von Probleminstanzen verschiedener Grösse und Komplexität.

Abstractions in probabilistic planningBachelor's thesis, February 2016. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
In planning what we want to do is to get from an initial state into a goal state. A state can be described by a finite number of boolean valued variables. If we want to transition from one state to the other we have to apply an action and this, at least in probabilistic planning, leads to a probability distribution over a set of possible successor states. From each transition the agent gains a reward dependent on the current state and his action. In this setting the growth of the number of possible states is exponential with the number of variables. We assume that the value of these variables is determined for each variable independently in a probabilistic fashion. So these variables influence the number of possible successor states in the same way as they did the state space. In consequence it is almost impossible to obtain an optimal amount of reward approaching this problem with a brute force technique. One way to get past this problem is to abstract the problem and then solve a simplified version of the aforementioned. That’s in general the idea proposed by Boutilier and Dearden [1]. They have introduced a method to create an abstraction which depends on the reward formula and the dependencies contained in the problem. With this idea as a basis we’ll create a heuristic for a trialbased heuristic tree search (THTS) algorithm [5] and a standalone planner using the framework PROST (Keller and Eyerich, 2012). These will then be tested on all the domains of the International Probabilistic Planning Competition (IPPC).

P^mKompilierung von Planungsaufgaben im FastDownwardPlanungssystem als alternative Berechnung der h^mHeuristikBachelor's thesis, January 2016. Download: (PDF) (slides; PDF) (sources; ZIP)
In einer Planungsaufgabe geht es darum einen gegebenen Wertezustand durch sequentielles Anwenden von Aktionen in einen Wertezustand zu überführen, welcher geforderte Zieleigenschaften erfüllt. Beim Lösen von Planungsaufgaben zählt Effizienz. Um Zeit und Speicher zu sparen verwenden viele Planer heuristische Suche. Dabei wird mittels einer Heuristik abgeschätzt, welche Aktion als nächstes angewendet werden soll um möglichst schnell in einen gewünschten Zustand zu gelangen.
In dieser Arbeit geht es darum, die von Haslum vorgeschlagene P^{m}Kompilierung für Planungsaufgaben zu implementieren und die h^{max}Heuristik auf dem kompilierten Problem gegen die h^{m}Heuristik auf dem originalen Problem zu testen. Die Implementation geschieht als Ergänzung zum FastDownwardPlanungssystem. Die Resultate der Tests zeigen, dass mittels der Kompilierung die Zahl der gelösten Probleme erhöht werden kann. Das Lösen eines kompilierten Problems mit der h^{max}Heuristik geschieht im allgemeinen mit selbiger Informationstiefe schneller als das Lösen des originalen Problems mit der h^{m}Heuristik. Diesen Zeitgewinn erkauft man sich mit einem höheren Speicherbedarf.

Evaluation of Regression Search and State Subsumption in Classical PlanningBachelor's thesis, July 2015. Download: (PDF) (slides; PDF) (sources; ZIP)
The objective of classical planning is to find a sequence of actions which begins in a given initial state and ends in a state that satisfies a given goal condition. A popular approach to solve classical planning problems is based on heuristic forward search algorithms. In contrast, regression search algorithms apply actions “backwards” in order to find a plan from a goal state to the initial state. Currently, regression search algorithms are somewhat unpopular, as the generation of partial states in a basic regression search often leads to a significant growth of the explored search space. To tackle this problem, state subsumption is a pruning technique that additionally discards newly generated partial states for which a more general partial state has already been explored.
In this thesis, we discuss and evaluate techniques of regression and state subsumption. In order to evaluate their performance, we have implemented a regression search algorithm for the planning system Fast Downward, supporting both a simple subsumption technique as well as a refined subsumption technique using a trie data structure. The experiments have shown that a basic regression search algorithm generally increases the number of explored states compared to uniformcost forward search. Regression with pruning based on state subsumption with a trie data structure significantly reduces the number of explored states compared to basic regression.

Solving the Traveling Tournament Problem with Heuristic SearchBachelor's thesis, February 2015. Download: (PDF) (slides; PDF) (sources; ZIP)
This thesis discusses the Traveling Tournament Problem and how it can be solved with heuristic search. The Traveling Tournament problem is a sports scheduling problem where one tries to find a schedule for a league that meets certain constraints while minimizing the overall distance traveled by the teams in this league. It is hard to solve for leagues with many teams involved since its complexity grows exponentially in the number of teams. The largest instances solved up to date, are instances with leagues of up to 10 teams.
Previous related work has shown that it is a reasonable approach to solve the Traveling Tournament Problem with an IDA*based tree search. In this thesis I implemented such a search and extended it with several enhancements to examine whether they improve performance of the search. The heuristic that I used in my implementation is the Independent Lower Bound heuristic. It tries to find lower bounds to the traveling costs of each team in the considered league. With my implementation I was able to solve problem instances with up to 8 teams. The results of my evaluation have mostly been consistent with the expected impact of the implemented enhancements on the overall performance.

TunnelBased Pruning for Classical PlanningBachelor's thesis, February 2015. Download: (PDF) (slides; PDF) (sources; ZIP)
One huge topic in Artificial Intelligence is the classical planning. It is the process of finding a plan, therefore a sequence of actions that leads from an initial state to a goal state for a specified problem. In problems with a huge amount of states it is very difficult and time consuming to find a plan. There are different pruning methods that attempt to lower the amount of time needed to find a plan by trying to reduce the number of states to explore. In this work we take a closer look at two of these pruning methods. Both of these methods rely on the last action that led to the current state. The first one is the so called tunnel pruning that is a generalisation of the tunnel macros that are used to solve Sokoban problems. The idea is to find actions that allow a tunnel and then prune all actions that are not in the tunnel of this action. The second method is the partitionbased path pruning. In this method all actions are distributed into different partitions. These partitions then can be used to prune actions that do not belong to the current partition.
The evaluation of these two pruning methods show, that they can reduce the number of explored states for some problem domains, however the difference between pruned search and normal search gets smaller when we use heuristic functions. It also shows that the two pruning rules effect different problem domains.

Kontextbasierte Suche für klassische HandlungsplanungBachelor's thesis, August 2014. Download: (PDF) (slides; PDF) (sources; ZIP)
Ziel klassischer Handlungsplanung ist es auf eine möglichst effiziente Weise gegebene Planungsprobleme zu lösen. Die Lösung bzw. der Plan eines Planungsproblems ist eine Sequenz von Operatoren mit denen man von einem Anfangszustand in einen Zielzustand gelangt. Um einen Zielzustand gezielter zu finden, verwenden einige Suchalgorithmen eine zusätzliche Information über den Zustandsraum  die Heuristik. Sie schätzt, ausgehend von einem Zustand den Abstand zum Zielzustand. Demnach wäre es ideal, wenn jeder neue besuchte Zustand einen kleineren heuristischen Wert aufweisen würde als der bisher besuchte Zustand. Es gibt allerdings Suchszenarien bei denen die Heuristik nicht weiterhilft um einem Ziel näher zu kommen. Dies ist insbesondere dann der Fall, wenn sich der heuristische Wert von benachbarten Zuständen nicht ändert. Für die gierige Bestensuche würde das bedeuten, dass die Suche auf Plateaus und somit blind verläuft, weil sich dieser Suchalgorithmus ausschliesslich auf die Heuristik stützt. Algorithmen, die die Heuristik als Wegweiser verwenden, gehören zur Klasse der heuristischen Suchalgorithmen.
In dieser Arbeit geht es darum, in Fällen wie den Plateaus trotzdem eine Orientierung im Zustandsraum zu haben, indem Zustände neben der Heuristik einer weiteren Priorisierung unterliegen. Die hier vorgestellte Methode nutzt Abhängigkeiten zwischen Operatoren aus und erweitert die gierige Bestensuche. Wie stark Operatoren voneinander abhängen, betrachten wir anhand eines Abstandsmasses, welches vor der eigentlichen Suche berechnet wird. Die grundlegende Idee ist, Zustände zu bevorzugen, deren Operatoren im Vorfeld voneinander profitierten. Die Heuristik fungiert hierbei erst im Nachhinein als TieBreaker, sodass wir einem vielversprechenden Pfad zunächst folgen können, ohne dass uns die Heuristik an einer anderen, weniger vielversprechenden Stelle suchen lässt.
Die Ergebnisse zeigen, dass unser Ansatz in der reinen Suchzeit je nach Heuristik performanter sein kann, als wenn man sich ausschliesslich auf die Heuristik stützt. Bei sehr informationsreichen Heuristiken kann es jedoch passieren, dass die Suche durch unseren Ansatz eher gestört wird. Zudem werden viele Probleme nicht gelöst, weil die Berechnung der Abstände zu zeitaufwändig ist.

Generating and Evaluating Unsolvable STRIPS Planning Instances for Classical PlanningBachelor's thesis, August 2014. Download: (PDF) (slides; PDF) (sources; TAR.GZ)
In classical planning, heuristic search is a popular approach to solving problems very efficiently. The objective of planning is to find a sequence of actions that can be applied to a given problem and that leads to a goal state. For this purpose, there are many heuristics. They are often a big help if a problem has a solution, but what happens if a problem does not have one? Which heuristics can help proving unsolvability without exploring the whole state space? How efficient are they? Admissible heuristics can be used for this purpose because they never overestimate the distance to a goal state and are therefore able to safely cut off parts of the search space. This makes it potentially easier to prove unsolvability
In this project we developed a problem generator to automatically create unsolvable problem instances and used those generated instances to see how different admissible heuristics perform on them. We used the Japanese puzzle game Sokoban as the first problem because it has a high complexity but is still easy to understand and to imagine for humans. As second problem, we used a logistical problem called NoMystery because unlike Sokoban it is a resource constrained problem and therefore a good supplement to our experiments. Furthermore, unsolvability occurs rather 'naturally' in these two domains and does not seem forced.

Phase Transitions in the Solvability of SokobanBachelor's thesis, December 2013. Download: (PDF)
Sokoban is a computer game where each level consists of a twodimensional grid of fields. There are walls as obstacles, moveable boxes and goal fields. The player controls the warehouse worker (Sokoban in Japanese) to push the boxes to the goal fields. The problem is very complex and that is why Sokoban has become a domain in planning.
Phase transitions mark a sudden change in solvability when traversing through the problem space. They occur in the region of hard instances and have been found for many domains. In this thesis we investigate phase transitions in the Sokoban puzzle. For our investigation we generate and evaluate random instances. We declare the defining parameters for Sokoban and measure their influence on the solvability. We show that phase transitions in the solvability of Sokoban can be found and their occurrence is measured. We attempt to unify the parameters of Sokoban to get a prediction on the solvability and hardness of specific instances.

Iterative Tunneling A* in PlanningBachelor's thesis, July 2013. Download: (PDF)
In planning, we address the problem of automatically finding a sequence of actions that leads from a given initial state to a state that satisfies some goal condition. In satisficing planning, our objective is to find plans with preferably low, but not necessarily the lowest possible costs while keeping in mind our limited resources like time or memory. A prominent approach for satisficing planning is based on heuristic search with inadmissible heuristics. However, depending on the applied heuristic, plans found with heuristic search might be of low quality, and hence, improving the quality of such plans is often desirable. In this thesis, we adapt and apply iterative tunneling search with A* (ITSA*) to planning. ITSA* is an algorithm for plan improvement which has been originally proposed by Furcy et al. for search problems. ITSA* intends to search the local space of a given solution path in order to find "short cuts" which allow us to improve our solution. In this thesis, we provide an implementation and systematic evaluation of this algorithm on the standard IPC benchmarks. Our results show that ITSA* also successfully works in the planning area.

An Algorithm for Avoiding Plateaus in Heuristic SearchBachelor's thesis, June 2013. Download: (PDF)
In action planning, greedy bestfirst search (GBFS) is one of the standard techniques if suboptimal plans are accepted. GBFS uses a heuristic function to guide the search towards a goal state. To achieve generality, in domainindependant planning the heuristic function is generated automatically. A wellknown problem of GBFS are search plateaus, i.e., regions in the search space where all states have equal heuristic values. In such regions, heuristic search can degenerate to uninformed search. Hence, techniques to escape from such plateaus are desired to improve the efficiency of the search. A recent approach to avoid plateaus is based on diverse bestfirst search (DBFS) proposed by Imai and Kishimoto. However, this approach relies on several parameters. This thesis presents an implementation of DBFS into the Fast Downward planner. Furthermore, this thesis presents a systematic evaluation of DBFS for several parameter settings, leading to a better understanding of the impact of the parameter choices to the search performance.

A Learning AI for the game Risk using the TD(λ)AlgorithmBachelor's thesis, June 2013. Download: (PDF)
Risk is a popular board game where players conquer each other's countries. In this project, I created an AI that plays Risk and is capable of learning. For each decision it makes, it performs a simple search one step ahead, looking at the outcomes of all possible moves it could make, and picks the most beneficial. It judges the desirability of outcomes by a series of parameters, which are modified after each game using the TD(λ)Algorithm, allowing the AI to learn.

Optimal Policies for the Canadian Traveler's ProblemBachelor's thesis, October 2012.
The Canadian Traveler's Problem (ctp) is a path finding problem where due to unfavorable weather, some of the roads are impassable. At the beginning, the agent does not know which roads are traversable and which are not. Instead, it can observe the status of roads adjacent to its current location. We consider the stochastic variant of the problem, where the blocking status of a connection is randomly defined with known probabilities. The goal is to find a policy which minimizes the expected travel costs of the agent.
We discuss several properties of the stochastic ctp and present an efficient way to calculate state probabilities. With the aid of these theoretical results, we introduce an uninformed algorithm to find optimal policies.

A Pattern Database Approach for Solving the TopSpin Puzzle ProblemBachelor's thesis, August 2012. Download: (PDF)
Finding optimal solutions for general search problems is a challenging task. A powerful approach for solving such problems is based on heuristic search with pattern database heuristics. In this thesis, we present a domain specific solver for the TopSpin Puzzle problem. This solver is based on the abovementioned pattern database approach. We investigate several pattern databases, and evaluate them on problem instances of different size.

An Algorithm for Computing Bisimulations in PlanningBachelor's thesis, July 2012. Download: (PDF)
Mergeandshrink abstractions are a popular approach to generate abstraction heuristics for planning. The computation of mergeandshrink abstractions relies on a merging and a shrinking strategy. A recently investigated shrinking strategy is based on using bisimulations. Bisimulations are guaranteed to produce perfect heuristics. In this thesis, we investigate an efficient algorithm proposed by Dovier et al. for computing coarsest bisimulations. The algorithm, however, cannot directly be applied to planning and needs some adjustments. We show how this algorithm can be reduced to work with planning problems. In particular, we show how an edge labelled state space can be translated to a state labelled one and what other changes are necessary for the algorithm to be usable for planning problems. This includes a custom data structure to fulfil all requirements to meet the worst case complexity. Furthermore, the implementation will be evaluated on planning problems from the International Planning Competitions. We will see that the resulting algorithm can often not compete with the currently implemented algorithm in Fast Downward. We discuss the reasons why this is the case and propose possible solutions to resolve this issue.

CSP and SATbased Inference Techniques Applied to GnomineBachelor's thesis, July 2012. Download: (PDF)
In order to understand an algorithm, it is always helpful to have a visualization that shows step for step what the algorithm is doing. Under this presumption this Bachelor project will explain and visualize two AI techniques, Constraint Satisfaction Processing and SAT Backbones, using the game Gnomine as an example.
CSP techniques build up a network of constraints and infer information by propagating through a single or several constraints at a time, reducing the domain of the variables in the constraint(s). SAT Backbone Computations find literals in a propositional formula, which are true in every model of the given formula.
By showing how to apply these algorithms on the problem of solving a Gnomine game I hope to give a better insight on the nature of how the chosen algorithms work.

Refining Abstraction Heuristics with MutexesBachelor's thesis, July 2012. Download: (PDF)
Planning as heuristic search is a powerful approach to solve domainindependent planning problems. An important class of heuristics is based on abstractions of the original planning task. However, abstraction heuristics usually come with loss in precision. The contribution of this thesis is the investigation of constrained abstraction heuristics in general, and the application of this concept to pattern database and merge and shrink abstractions in particular. The idea is to use a subclass of mutexes which represent sets of variablevaluepairs so that only one of these pairs can be true at any given time, to regain some of the precision which is lost in the abstraction without increasing its size. By removing states and operators in the abstraction which conflict with such a mutex, the abstraction is refined and hence, the corresponding abstraction heuristic can get more informed. We have implemented the refinements of these heuristics in the Fast Downward planner and evaluated the different approaches using standard IPC benchmarks. The results show that the concept of constrained abstraction heuristics can improve planning as heuristic search in terms of time and coverage.

Search Methods for General Permutation ProblemsBachelor's thesis, July 2012. Download: (PDF)
A permutation problem considers the task where an initial order of objects (ie, an initial mapping of objects to locations) must be reordered into a given goal order by using permutation operators. Permutation operators are 1:1 mappings of the objects from their locations to (possibly other) locations. An example for permutation problems are the wellknown Rubik's Cube and TopSpin Puzzle. Permutation problems have been a research area for a while, and several methods for solving such problems have been proposed in the last two centuries. Most of these methods focused on finding optimal solutions, causing an exponential runtime in the worst case.
In this work, we consider an algorithm for solving permutation problems that has been originally proposed by M. Furst, J. Hopcroft and E. Luks in 1980. This algorithm has been introduced on a theoretical level within a proof for "Testing Membership and Determining the Order of a Group", but has not been implemented and evaluated on practical problems so far. In contrast to the other abovementioned solving algorithms, it only finds suboptimal solutions, but is guaranteed to run in polynomial time. The basic idea is to iteratively reach subgoals, and then to let them fix when we go further to reach the next goals. We have implemented this algorithm and evaluated it on different models, as the Pancake Problem and the TopSpin Puzzle .

Construction of Pattern Database Heuristics Using CostPartitioningBachelor's thesis, June 2012. Download: (PDF)
Pattern databases (Culberson & Schaeffer, 1998) or PDBs, have been proven very effective in creating admissible Heuristics for singleagent search, such as the A*algorithm. Haslum et. al proposed, a hillclimbing algorithm can be used to construct the PDBs, using the canonical heuristic. A different approach would be to change actioncosts in the patternrelated abstractions, in order to obtain the admissible heuristic. This the so called CostPartitioning.
The aim of this project was to implement a costpartitioning inside the hillclimbing algorithm by Haslum, and compare the results with the standard way which uses the canonical heuristic.

UCT for PacManBachelor's thesis, November 2011. Download: (PDF)
UCT ("upper confidence bounds applied to trees") is a stateoftheart algorithm for acting under uncertainty, e.g. in probabilistic environments. In the last years it has been very successfully applied in numerous contexts, including twoplayer board games like Go and Mancala and stochastic singleagent optimization problems such as path planning under uncertainty and probabilistic action planning.
In this project the UCT algorithm was implemented, adapted and evaluated for the classical arcade game "Ms PacMan". The thesis introduces Ms PacMan and the UCT algorithm, discusses some critical design decisions for developing a strong UCTbased algorithm for playing Ms PacMan, and experimentally evaluates the implementation.