bibin
/
cphb
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Improve language

This commit is contained in:
Antti H S Laaksonen 2017-05-18 20:03:10 +03:00
parent 69941cb3a1
commit d3c0e57236
1 changed files with 113 additions and 109 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

222
chapter05.tex
View File

@ -25,15 +25,15 @@ all subsets of a set of $n$ elements.
For example, the subsets of $\{0,1,2\}$ are For example, the subsets of $\{0,1,2\}$ are
$\emptyset$, $\{0\}$, $\{1\}$, $\{2\}$, $\{0,1\}$, $\emptyset$, $\{0\}$, $\{1\}$, $\{2\}$, $\{0,1\}$,
$\{0,2\}$, $\{1,2\}$ and $\{0,1,2\}$. $\{0,2\}$, $\{1,2\}$ and $\{0,1,2\}$.
There are two common methods for this: There are two common methods to generate subsets:
we can either implement a recursive search we can either perform a recursive search
or use bit operations of integers. or exploit the bit representation of integers.
\subsubsection{Method 1} \subsubsection{Method 1}
An elegant way to go through all subsets An elegant way to go through all subsets
of a set is to use recursion. of a set is to use recursion.
The following function The following function \texttt{search}
generates the subsets of the set generates the subsets of the set
$\{0,1,\ldots,n-1\}$. $\{0,1,\ldots,n-1\}$.
The function maintains a vector \texttt{subset} The function maintains a vector \texttt{subset}
@ -42,28 +42,29 @@ The search begins when the function is called
with parameter 0. with parameter 0.
\begin{lstlisting} \begin{lstlisting}
void gen(int k) { void search(int k) {
if (k == n) { if (k == n) {
// process subset // process subset
} else { } else {
gen(k+1); search(k+1);
subset.push_back(k); subset.push_back(k);
gen(k+1); search(k+1);
subset.pop_back(); subset.pop_back();
} }
} }
\end{lstlisting} \end{lstlisting}
The parameter $k$ is the next When the function \texttt{search}
candidate to be included in the subset. is called with parameter $k$,
The function considers two cases that both it decides whether to include
generate a recursive call: element $k$ in the subset or not,
either $k$ is included or not included in the subset. and in both cases,
Finally, when $k=n$, all elements have been processed then calls itself with parameter $k+1$
and one subset has been generated. However, if $k=n$, the function notices that
all elements have been processed
and a subset has been generated.
The following tree illustrates how the function is The following tree illustrates the function calls when $n=3$.
called when $n=3$.
We can always choose either the left branch We can always choose either the left branch
($k$ is not included in the subset) or the right branch ($k$ is not included in the subset) or the right branch
($k$ is included in the subset). ($k$ is included in the subset).
@ -72,32 +73,32 @@ We can always choose either the left branch
\begin{tikzpicture}[scale=.45] \begin{tikzpicture}[scale=.45]
\begin{scope} \begin{scope}
\small \small
\node at (0,0) {$\texttt{gen}(0)$}; \node at (0,0) {$\texttt{search}(0)$};
\node at (-8,-4) {$\texttt{gen}(1)$}; \node at (-8,-4) {$\texttt{search}(1)$};
\node at (8,-4) {$\texttt{gen}(1)$}; \node at (8,-4) {$\texttt{search}(1)$};
\path[draw,thick,->] (0,0-0.5) -- (-8,-4+0.5); \path[draw,thick,->] (0,0-0.5) -- (-8,-4+0.5);
\path[draw,thick,->] (0,0-0.5) -- (8,-4+0.5); \path[draw,thick,->] (0,0-0.5) -- (8,-4+0.5);
\node at (-12,-8) {$\texttt{gen}(2)$}; \node at (-12,-8) {$\texttt{search}(2)$};
\node at (-4,-8) {$\texttt{gen}(2)$}; \node at (-4,-8) {$\texttt{search}(2)$};
\node at (4,-8) {$\texttt{gen}(2)$}; \node at (4,-8) {$\texttt{search}(2)$};
\node at (12,-8) {$\texttt{gen}(2)$}; \node at (12,-8) {$\texttt{search}(2)$};
\path[draw,thick,->] (-8,-4-0.5) -- (-12,-8+0.5); \path[draw,thick,->] (-8,-4-0.5) -- (-12,-8+0.5);
\path[draw,thick,->] (-8,-4-0.5) -- (-4,-8+0.5); \path[draw,thick,->] (-8,-4-0.5) -- (-4,-8+0.5);
\path[draw,thick,->] (8,-4-0.5) -- (4,-8+0.5); \path[draw,thick,->] (8,-4-0.5) -- (4,-8+0.5);
\path[draw,thick,->] (8,-4-0.5) -- (12,-8+0.5); \path[draw,thick,->] (8,-4-0.5) -- (12,-8+0.5);
\node at (-14,-12) {$\texttt{gen}(3)$}; \node at (-14,-12) {$\texttt{search}(3)$};
\node at (-10,-12) {$\texttt{gen}(3)$}; \node at (-10,-12) {$\texttt{search}(3)$};
\node at (-6,-12) {$\texttt{gen}(3)$}; \node at (-6,-12) {$\texttt{search}(3)$};
\node at (-2,-12) {$\texttt{gen}(3)$}; \node at (-2,-12) {$\texttt{search}(3)$};
\node at (2,-12) {$\texttt{gen}(3)$}; \node at (2,-12) {$\texttt{search}(3)$};
\node at (6,-12) {$\texttt{gen}(3)$}; \node at (6,-12) {$\texttt{search}(3)$};
\node at (10,-12) {$\texttt{gen}(3)$}; \node at (10,-12) {$\texttt{search}(3)$};
\node at (14,-12) {$\texttt{gen}(3)$}; \node at (14,-12) {$\texttt{search}(3)$};
\node at (-14,-13.5) {$\emptyset$}; \node at (-14,-13.5) {$\emptyset$};
\node at (-10,-13.5) {$\{2\}$}; \node at (-10,-13.5) {$\{2\}$};
@ -123,7 +124,7 @@ We can always choose either the left branch
\subsubsection{Method 2} \subsubsection{Method 2}
Another way to generate subsets is to exploit Another way to generate subsets is based on
the bit representation of integers. the bit representation of integers.
Each subset of a set of $n$ elements Each subset of a set of $n$ elements
can be represented as a sequence of $n$ bits, can be represented as a sequence of $n$ bits,
@ -131,13 +132,14 @@ which corresponds to an integer between $0 \ldots 2^n-1$.
The ones in the bit sequence indicate The ones in the bit sequence indicate
which elements are included in the subset. which elements are included in the subset.
The usual convention is that the $k$th element The usual convention is that
is included in the subset exactly when the $k$th last bit the last bit corresponds to element 0,
in the sequence is one. the second last bit corresponds to element 1,
and so on.
For example, the bit representation of 25 For example, the bit representation of 25
is 11001, that corresponds to the subset $\{0,3,4\}$. is 11001, that corresponds to the subset $\{0,3,4\}$.
The following code goes through all subsets The following code goes through the subsets
of a set of $n$ elements of a set of $n$ elements
\begin{lstlisting} \begin{lstlisting}
@ -165,7 +167,7 @@ for (int b = 0; b < (1<<n); b++) {
\index{permutation} \index{permutation}
Next we will consider the problem of generating Next we consider the problem of generating
all permutations of a set of $n$ elements. all permutations of a set of $n$ elements.
For example, the permutations of $\{0,1,2\}$ are For example, the permutations of $\{0,1,2\}$ are
$(0,1,2)$, $(0,2,1)$, $(1,0,2)$, $(1,2,0)$, $(0,1,2)$, $(0,2,1)$, $(1,0,2)$, $(1,2,0)$,
@ -178,35 +180,35 @@ permutations iteratively.
Like subsets, permutations can be generated Like subsets, permutations can be generated
using recursion. using recursion.
The following function goes The following function \texttt{search} goes
through the permutations of the set $\{0,1,\ldots,n-1\}$. through the permutations of the set $\{0,1,\ldots,n-1\}$.
The function builds a vector \texttt{perm} that contains The function builds a vector \texttt{permutation}
the elements in the permutation, that describes the permutation,
and the search begins when the function is and the search begins when the function is
called without parameters. called without parameters.
\begin{lstlisting} \begin{lstlisting}
void gen() { void search() {
if (perm.size() == n) { if (permutation.size() == n) {
// process permutation // process permutation
} else { } else {
for (int i = 0; i < n; i++) { for (int i = 0; i < n; i++) {
if (chosen[i]) continue; if (chosen[i]) continue;
chosen[i] = true; chosen[i] = true;
perm.push_back(i); permutation.push_back(i);
gen(); search();
chosen[i] = false; chosen[i] = false;
perm.pop_back(); permutation.pop_back();
} }
} }
} }
\end{lstlisting} \end{lstlisting}
Each function call adds a new element to Each function call adds a new element to
the vector \texttt{perm}. \texttt{permutation}.
The array \texttt{chosen} indicates which The array \texttt{chosen} indicates which
elements are already included in the permutation. elements are already included in the permutation.
If the size of \texttt{perm} equals the size of the set, If the size of \texttt{permutation} equals the size of the set,
a permutation has been generated. a permutation has been generated.
\subsubsection{Method 2} \subsubsection{Method 2}
@ -215,20 +217,20 @@ a permutation has been generated.
Another method for generating permutations Another method for generating permutations
is to begin with the permutation is to begin with the permutation
$\{1,2,\ldots,n\}$ and repeatedly $\{0,1,\ldots,n-1\}$ and repeatedly
use a function that constructs the next permutation use a function that constructs the next permutation
in increasing order. in increasing order.
The C++ standard library contains the function The C++ standard library contains the function
\texttt{next\_permutation} that can be used for this: \texttt{next\_permutation} that can be used for this:
\begin{lstlisting} \begin{lstlisting}
vector<int> perm; vector<int> permutation;
for (int i = 0; i < n; i++) { for (int i = 0; i < n; i++) {
perm.push_back(i); permutation.push_back(i);
} }
do { do {
// process permutation // process permutation
} while (next_permutation(perm.begin(),perm.end())); } while (next_permutation(permutation.begin(),permutation.end()));
\end{lstlisting} \end{lstlisting}
\section{Backtracking} \section{Backtracking}
@ -244,13 +246,13 @@ a solution can be constructed.
\index{queen problem} \index{queen problem}
As an example, consider the \key{queen problem} As an example, consider the problem of
where the task is to calculate the number calculating the number
of ways we can place $n$ queens to of ways $n$ queens can be placed to
an $n \times n$ chessboard so that an $n \times n$ chessboard so that
no two queens attack each other. no two queens attack each other.
For example, when $n=4$, For example, when $n=4$,
there are two possible solutions to the problem: there are two possible solutions:
\begin{center} \begin{center}
\begin{tikzpicture}[scale=.65] \begin{tikzpicture}[scale=.65]
@ -322,23 +324,24 @@ the backtracking algorithm are as follows:
\draw (-1,-6) -- (1,-8); \draw (-1,-6) -- (1,-8);
\draw (-1,-6) -- (6,-8); \draw (-1,-6) -- (6,-8);
\node at (-9,-13) {\ding{55}}; \node at (-9,-13) {illegal};
\node at (-4,-13) {\ding{55}}; \node at (-4,-13) {illegal};
\node at (1,-13) {\ding{55}}; \node at (1,-13) {illegal};
\node at (6,-13) {\ding{51}}; \node at (6,-13) {valid};
\end{scope} \end{scope}
\end{tikzpicture} \end{tikzpicture}
\end{center} \end{center}
At the bottom level, the three first boards At the bottom level, the three first configurations
are not valid, because the queens attack each other. are illegal, because the queens attack each other.
However, the fourth board is valid However, the fourth configuration is valid
and it can be extended to a complete solution by and it can be extended to a complete solution by
placing two more queens to the board. placing two more queens to the board.
There is only one way to place the two remaining queens.
\begin{samepage} \begin{samepage}
The following code implements the search: The algorithm can be implemented as follows:
\begin{lstlisting} \begin{lstlisting}
void search(int y) { void search(int y) {
if (y == n) { if (y == n) {
@ -360,11 +363,13 @@ and the code calculates the number of solutions
to \texttt{count}. to \texttt{count}.
The code assumes that the rows and columns The code assumes that the rows and columns
of the board are numbered from 0. of the board are numbered from 0 to $n-1$.
The function places a queen to row $y$ When the function \texttt{search} is
where $0 \le y < n$. called with parameter $y$,
Finally, if $y=n$, a solution has been found it places a queen to row $y$
and the variable $c$ is increased by one. and then calls itself with parameter $y+1$.
However, if $y=n$, a solution has been found
and the variable \texttt{count} is increased by one.
The array \texttt{r1} keeps track of the columns The array \texttt{r1} keeps track of the columns
that already contain a queen, that already contain a queen,
@ -372,7 +377,7 @@ and the arrays \texttt{r2} and \texttt{r3}
keep track of the diagonals. keep track of the diagonals.
It is not allowed to add another queen to a It is not allowed to add another queen to a
column or diagonal that already contains a queen. column or diagonal that already contains a queen.
For example, the rows and diagonals of For example, the columns and diagonals of
the $4 \times 4$ board are numbered as follows: the $4 \times 4$ board are numbered as follows:
\begin{center} \begin{center}
@ -467,8 +472,8 @@ effect on the efficiency of the search.
Let us consider the problem Let us consider the problem
of calculating the number of paths of calculating the number of paths
in an $n \times n$ grid from the upper-left corner in an $n \times n$ grid from the upper-left corner
to the lower-right corner so that each square to the lower-right corner such that the
will be visited exactly once. path visits each square exactly once.
For example, in a $7 \times 7$ grid, For example, in a $7 \times 7$ grid,
there are 111712 such paths. there are 111712 such paths.
One of the paths is as follows: One of the paths is as follows:
@ -489,14 +494,14 @@ One of the paths is as follows:
\end{tikzpicture} \end{tikzpicture}
\end{center} \end{center}
We will concentrate on the $7 \times 7$ case, We focus on the $7 \times 7$ case,
because its level of difficulty is appropriate to our needs. because its level of difficulty is appropriate to our needs.
We begin with a straightforward backtracking algorithm, We begin with a straightforward backtracking algorithm,
and then optimize it step by step using observations and then optimize it step by step using observations
how the search can be pruned. of how the search can be pruned.
After each optimization, we measure the running time After each optimization, we measure the running time
of the algorithm and the number of recursive calls, of the algorithm and the number of recursive calls,
so that we will clearly see the effect of each so that we clearly see the effect of each
optimization on the efficiency of the search. optimization on the efficiency of the search.
\subsubsection{Basic algorithm} \subsubsection{Basic algorithm}
@ -557,8 +562,8 @@ For example, the following paths are symmetric:
\end{center} \end{center}
Hence, we can decide that we always first Hence, we can decide that we always first
move one step down, move one step down (or right),
and finally multiply the number of the solutions by two. and finally multiply the number of solutions by two.
\begin{itemize} \begin{itemize}
\item \item
@ -598,12 +603,12 @@ number of recursive calls: 20 billion
\subsubsection{Optimization 3} \subsubsection{Optimization 3}
If the path touches the wall so that there is If the path touches a wall
an unvisited square on both sides, and can turn either left or right,
the grid splits into two parts. the grid splits into two parts
For example, in the following path that contain unvisited squares.
both the left and right squares For example, in the following situation,
are unvisited: the path can turn either left or right:
\begin{center} \begin{center}
\begin{tikzpicture}[scale=.55] \begin{tikzpicture}[scale=.55]
@ -614,12 +619,15 @@ are unvisited:
(3.5,0.5) -- (3.5,1.5) -- (1.5,1.5) -- (3.5,0.5) -- (3.5,1.5) -- (1.5,1.5) --
(1.5,2.5) -- (4.5,2.5) -- (4.5,0.5) -- (1.5,2.5) -- (4.5,2.5) -- (4.5,0.5) --
(5.5,0.5) -- (5.5,6.5); (5.5,0.5) -- (5.5,6.5);
\node at (4.5,6.5) {$a$};
\node at (6.5,6.5) {$b$};
\end{scope} \end{scope}
\end{tikzpicture} \end{tikzpicture}
\end{center} \end{center}
Now it will not be possible to visit every square, In this case, we cannot visit all squares anymore,
so we can terminate the search. so we can terminate the search.
It turns out that this optimization is very useful: This optimization is very useful:
\begin{itemize} \begin{itemize}
\item \item
@ -630,18 +638,14 @@ number of recursive calls: 221 million
\subsubsection{Optimization 4} \subsubsection{Optimization 4}
The idea of the previous optimization The idea of Optimization 3
can be generalized: can be generalized:
if the path cannot continue forward
but can turn either left or right,
the grid splits into two parts the grid splits into two parts
if the top and bottom neighbors that both contain unvisited squares.
of the current square are unvisited and For example, consider the following path:
the left and right neighbors are
wall or visited (or vice versa).
For example, in the following path
the top and bottom neighbors are unvisited,
so the path cannot visit all squares
in the grid anymore:
\begin{center} \begin{center}
\begin{tikzpicture}[scale=.55] \begin{tikzpicture}[scale=.55]
\begin{scope} \begin{scope}
@ -654,8 +658,9 @@ in the grid anymore:
\end{scope} \end{scope}
\end{tikzpicture} \end{tikzpicture}
\end{center} \end{center}
Thus, we can terminate the search in all such cases. It is clear that we cannot visit all squares anymore,
After this optimization, the search will be so we can terminate the search.
After this optimization, the search is
very efficient: very efficient:
\begin{itemize} \begin{itemize}
@ -673,7 +678,7 @@ was 483 seconds,
and now after the optimizations, and now after the optimizations,
the running time is only 0.6 seconds. the running time is only 0.6 seconds.
Thus, the algorithm became nearly 1000 times Thus, the algorithm became nearly 1000 times
faster thanks to the optimizations. faster after the optimizations.
This is a usual phenomenon in backtracking, This is a usual phenomenon in backtracking,
because the search tree is usually large because the search tree is usually large
@ -705,8 +710,8 @@ middle technique.
As an example, consider a problem where As an example, consider a problem where
we are given a list of $n$ numbers and we are given a list of $n$ numbers and
a number $x$. a number $x$,
Our task is to find out if it is possible and we want to find out if it is possible
to choose some numbers from the list so that to choose some numbers from the list so that
their sum is $x$. their sum is $x$.
For example, given the list $[2,4,5,9]$ and $x=15$, For example, given the list $[2,4,5,9]$ and $x=15$,
@ -717,11 +722,11 @@ it is not possible to form the sum.
An easy solution to the problem is to An easy solution to the problem is to
go through all subsets of the elements and go through all subsets of the elements and
check if the sum of any of the subsets is $x$. check if the sum of any of the subsets is $x$.
The running time of such a solution is $O(2^n)$, The running time of such an algorithm is $O(2^n)$,
because there are $2^n$ subsets. because there are $2^n$ subsets.
However, using the meet in the middle technique, However, using the meet in the middle technique,
we can achieve a more efficient $O(2^{n/2})$ time solution\footnote{This we can achieve a more efficient $O(2^{n/2})$ time algorithm\footnote{This
technique was introduced in 1974 by E. Horowitz and S. Sahni \cite{hor74}.}. idea was introduced in 1974 by E. Horowitz and S. Sahni \cite{hor74}.}.
Note that $O(2^n)$ and $O(2^{n/2})$ are different Note that $O(2^n)$ and $O(2^{n/2})$ are different
complexities because $2^{n/2}$ equals $\sqrt{2^n}$. complexities because $2^{n/2}$ equals $\sqrt{2^n}$.
@ -729,29 +734,28 @@ The idea is to divide the list into
two lists $A$ and $B$ such that both two lists $A$ and $B$ such that both
lists contain about half of the numbers. lists contain about half of the numbers.
The first search generates all subsets The first search generates all subsets
of the numbers in $A$ and stores their sums of $A$ and stores their sums to a list $S_A$.
to a list $S_A$.
Correspondingly, the second search creates Correspondingly, the second search creates
a list $S_B$ from $B$. a list $S_B$ from $B$.
After this, it suffices to check if it is possible After this, it suffices to check if it is possible
to choose one element from $S_A$ and another to choose one element from $S_A$ and another
element from $S_B$ such that their sum is $x$. element from $S_B$ such that their sum is $x$.
This is possible exactly when there is a way to This is possible exactly when there is a way to
form the sum $x$ using the numbers in the original list. form the sum $x$ using the numbers of the original list.
For example, suppose that the list is $[2,4,5,9]$ and $x=15$. For example, suppose that the list is $[2,4,5,9]$ and $x=15$.
First, we divide the list into $A=[2,4]$ and $B=[5,9]$. First, we divide the list into $A=[2,4]$ and $B=[5,9]$.
After this, we create lists After this, we create lists
$S_A=[0,2,4,6]$ and $S_B=[0,5,9,14]$. $S_A=[0,2,4,6]$ and $S_B=[0,5,9,14]$.
In this case, the sum $x=15$ is possible to form, In this case, the sum $x=15$ is possible to form,
because we can choose the number $6$ from $S_A$ because $S_A$ contains the sum $6$,
and the number $9$ from $S_B$, $S_B$ contains the sum $9$, and $6+9=15$.
which corresponds to the solution $[2,4,9]$. This corresponds to the solution $[2,4,9]$.
The time complexity of the algorithm is $O(2^{n/2})$, The time complexity of the algorithm is $O(2^{n/2})$,
because both lists $A$ and $B$ contain about $n/2$ numbers because both lists $A$ and $B$ contain about $n/2$ numbers
and it takes $O(2^{n/2})$ time to calculate the sums of and it takes $O(2^{n/2})$ time to calculate the sums of
their subsets to lists $S_A$ and $S_B$. their subsets to lists $S_A$ and $S_B$.
After this, it is possible to check in After this, it is possible to check in
$O(2^{n/2})$ time if the sum $x$ can be formed $O(2^{n/2})$ time if the sum $x$ can be created
using the numbers in $S_A$ and $S_B$. from $S_A$ and $S_B$.