%\documentstyle[10pt,twoside]{article}
%\documentstyle[twoside]{article}
\documentclass[twoside]{article}
\setlength{\oddsidemargin}{0 in}
\setlength{\evensidemargin}{0 in}
\setlength{\topmargin}{-0.6 in}
\setlength{\textwidth}{6.7 in}
\setlength{\textheight}{8.5 in}
\setlength{\headsep}{0.75 in}
\setlength{\parindent}{0 in}
\setlength{\parskip}{0.1 in}
\usepackage{amsmath,amssymb,enumerate,algorithm,ifthen,float}
\usepackage[noend]{algorithmic}

% The following commands sets up the lecnum (lecture number)
% counter and make various numbering schemes work relative
% to the lecture number. Don't edit them!
%
\newcounter{lecnum}
\renewcommand{\thepage}{\thelecnum-\arabic{page}}
\renewcommand{\thesection}{\thelecnum.\arabic{section}}
\renewcommand{\theequation}{\thelecnum.\arabic{equation}}
\renewcommand{\thefigure}{\thelecnum.\arabic{figure}}
\renewcommand{\thetable}{\thelecnum.\arabic{table}}

\newtheorem{theorem}{Theorem} 
\newtheorem{lemma}{Lemma} 
\newtheorem{claim}{Claim} 
\newtheorem{proposition}{Proposition} 
\newtheorem{prob}{Problem} 
\newtheorem{corollary}{Corollary} 
\newtheorem{question}{Question} 
\newtheorem{conjecture}{Conjecture} 
\newtheorem{example}{Example} 
\newtheorem{definition}{Definition} 
\newtheorem{remarka}{Remark} 

\def\P{\mathop{\rm P}\nolimits}
\def\NP{\mathop{\rm NP}\nolimits}
\def\DTIME{\mathop{\rm DTIME}\nolimits}
\def\BPTIME{\mathop{\rm BPTIME}\nolimits}
\def\ZPTIME{\mathop{\rm ZPTIME}\nolimits}
\def\polylog{\mathop{\rm polylog}\nolimits}

\newenvironment{remark}{\begin{remarka}\rm}{\end{remarka}} 
\newenvironment{proof}{{\bf Proof.}}{\hfill\rule{2mm}{2mm}} 
\newenvironment{pproof}[1]{\noindent{\textbf{Proof of #1.}}}{\hfill\rule{2mm}{2mm}} 
\newcommand{\calI}{{\cal I}}
\newcommand{\calT}{{\cal T}}
\newcommand{\calP}{{\cal P}}
\newcommand{\opt}{\mbox{\sc opt}}
\newcommand{\OPT}{\mbox{\sc OPT}}
\newcommand{\QQ}{\mathbb{Q}}
\newcommand{\RR}{\mathbb{R}}
\newcommand{\ZZ}{\mathbb{Z}}



%
% The following macro is used to generate the header.
%
\newcommand{\lecture}[5]{
   \pagestyle{myheadings}
   \thispagestyle{plain}
   \newpage
   \setcounter{lecnum}{#1}
   \setcounter{page}{1}
   \noindent
   \begin{center}
   \framebox{
      \vbox{\vspace{2mm}
    \hbox to 6.28in { {\bf CMPUT 675: Approximation Algorithms
                        \hfill Fall 2014} }
       \vspace{4mm}
       \hbox to 6.28in { {\Large \hfill Lecture #1 (#2): #3 \hfill} }
       \vspace{2mm}
       \hbox to 6.28in { {\it Lecturer: #4 \hfill Scribe: #5} }
      \vspace{2mm}}
   }
   \end{center}
   \markboth{Lecture #1: #3}{Lecture #1: #3}
   \vspace*{4mm}
}

%
% Convention for citations is authors' initials followed by the year.
% For example, to cite a paper by Leighton and Maggs you would type
% \cite{LM89}, and to cite a paper by Strassen you would type \cite{S69}.
% (To avoid bibliography problems, for now we redefine the \cite command.)
%
\renewcommand{\cite}[1]{[#1]}

\input{epsf}

%Use this command for a figure; it puts a figure in wherever you want it.
%usage: \fig{NUMBER}{FIGURE-SIZE}{CAPTION}{FILENAME}
\newcommand{\fig}[4]{
			\vspace{0.2 in}
			\setlength{\epsfxsize}{#2}
			\centerline{\epsfbox{#4}}
			\begin{center}
			Figure \thelecnum.#1:~#3
			\end{center}
	}

% Use these for theorems, lemmas, proofs, etc.

% Some useful equation alignment commands, borrowed from TeX
\makeatletter
\def\eqalign#1{\,\vcenter{\openup\jot\m@th
  \ialign{\strut\hfil$\displaystyle{##}$&$\displaystyle{{}##}$\hfil
      \crcr#1\crcr}}\,}
\def\eqalignno#1{\displ@y \tabskip\@centering
  \halign to\displaywidth{\hfil$\displaystyle{##}$\tabskip\z@skip
    &$\displaystyle{{}##}$\hfil\tabskip\@centering
    &\llap{$##$}\tabskip\z@skip\crcr
    #1\crcr}}
\def\leqalignno#1{\displ@y \tabskip\@centering
  \halign to\displaywidth{\hfil$\displaystyle{##}$\tabskip\z@skip
    &$\displaystyle{{}##}$\hfil\tabskip\@centering
    &\kern-\displaywidth\rlap{$##$}\tabskip\displaywidth\crcr
    #1\crcr}}
\makeatother

% **** IF YOU WANT TO DEFINE ADDITIONAL MACROS FOR YOURSELF, PUT THEM HERE:

% NOW START EDITING
\begin{document}
%FILL IN THE RIGHT INFO.
%\lecture{**LECTURE-NUMBER**}{**DATE**}{**LECTURER**}{**SCRIBE**}
\lecture{28}{Nov 14}{Minimizing Makespan on Unrelated Machines}{Zachary Friggstad}{Antonio Carlos Salzvedel Furtado Junior}

% **** YOUR NOTES GO HERE:

\section{Minimizing Makespan on  Unrelated Machines}

In the problem \textsc{Minimizing Makespan on Unrelated Machines}, we are given a set of jobs $J = \{1,...,n\}$ and a set of machines $M = \{1,...,m\}$. Each job $j \in J$ requires processing time $p_{ij} \geq 0$ to be run on machine $i \in M$. Our objective is to find an assignment function $\phi : J \rightarrow M$ that minimizes the makespan:
$$
\max_{i \in M} \sum_{j: \phi (j) = i} p_{ij}
$$

Unlike the case of identical machines from an earlier lecture, there is probably no PTAS for this problem.
\begin{theorem}
For any $c < \frac{3}{2}$, there is no $c$-approximation for the \textsc{Minimizing the Makespan on Unrelated Machines}, unless ${\bf P}={\bf NP}$.
\end{theorem}
The proof is simple, but we skip it for the sake of time.

In this lecture, we saw an iterative rounding algorithm that provides a $2$-approximation for this problem. Our presentation follows
the approximation for the \textsc{Generalized Assignment Problem} that is recorded in \cite{LRS11}, but we focus on the
makespan minimization version for simplicity.

Consider the following LP relaxation for the problem. Here $\gamma$ represents the makespan and $x_{ij}$ is a binary variable that indicates if job $j$ is assigned to machine $i$.

\[
\begin{array}{rrcll}\tag{\bf LP-1}\label{lp:1}
{\rm minimize}: & \gamma \\
{\rm subject~to}: & \displaystyle \sum_{i \in M} x_{ij} = 1& \forall j \in J \\
& \displaystyle \sum_{j \in J} p_{ij} \cdot x_{ij} \leq \gamma & \forall i \in M \\
& x,\gamma \geq 0
\end{array}
\]

Unfortunately, (\ref{lp:1}) has a bad integrality gap. For example, consider the instance with jobs $J = \{1\}$ and machines $M=\{1,...,m\}$ such that $p_{i1} = m$ for each $i \in M$.
A feasible solution to (\ref{lp:1}) with value 1 is $\overline x_{i1} = 1/m$ for each $i \in M$ and $\gamma = 1$.
However, the optimal solution $OPT = m$,  since $m$ is the processing time any machine will take to run the only job.
Despite this bad gap, (\ref{lp:1}) will still be useful in our approximation after we strengthen it a bit.

The 2-approximation follows the same basic approach as the PTAS for identical machines: ``guess'' the target makespan with a binary search.
\begin{theorem}\label{thm:guess}
There is a polynomial-time algorithm that, given a value $T$, either returns a solution with makespan at most $2T$ or else reports there is no solution.
If $T \geq OPT$, it is guaranteed to find a solution with makespan at most $2T$.
\end{theorem}


The main tool used to prove Theorem \ref{thm:guess} is the strengthening (\ref{lp:feasibility}) of (\ref{lp:1}).
Note that it is not an LP in the strictest sense of the definition as there is no objective function; we are only interested in whether there is a feasible solution.
\[
\begin{array}{rrcll}\tag{\bf Feasibility-LP}\label{lp:feasibility}
& \displaystyle \sum_{i \in M} x_{ij} = 1 &\forall j \in J \\
& \displaystyle \sum_{j \in J} p_{ij} \cdot x_{ij} \leq T &\forall i \in M \\
& x_{ij} = 0 &\text{ if } p_{ij} > T \\
& x \geq 0
\end{array}
\]
In our bad example above, (\ref{lp:feasibility}) has no feasible solution if $T < m$.

The rest of this lecture is devoted to proving the following statement.
\begin{theorem}
If~(\ref{lp:feasibility}) has no feasible solution then $OPT > T$. If~(\ref{lp:feasibility}) has a feasible solution, then we can find an integer solution with makespan $\leq 2T$ in polynomial time.
\end{theorem}
Clearly when $T \geq OPT$ then the natural $\{0,1\}$ solution corresponding to the optimum is a feasible LP solution. So, we focus on proving that if there is a feasible solution
then we can find an assignment $\phi : J \rightarrow M$ with makespan at most $2T$.


In the course of the algorithm, we will be dropping some potential $j \rightarrow i$ assignments and we will also be removing some machines from consideration (after assigning them some jobs).
So, it will be convenient to view the problem in a more general setting.

Let $G = ( J \cup M, E)$ be a bipartite graph and, for each $i \in M$, let $T_{i}$ be a bound on the target running time of machine $i$. Now consider the following more general
feasibility LP.
\[
\begin{array}{rrcll}\tag{\bf Feasibility-LP2}\label{lp:feasibility2}
& \displaystyle \sum_{i : (i,j) \in E} x_{ij} = 1 &\forall j \in J \\
& \displaystyle \sum_{j : (i,j) \in E} p_{ij} \cdot x_{ij} \leq T_i &\forall i \in M \\
& x \geq 0
\end{array}
\]

%We will prove the following claim.
%\begin{claim}
%If~(\ref{lp:feasibility2}) has a feasible solution, then there is an assignment $\phi : J \rightarrow M$, such that machine $i$ is assigned $\leq T_{i} + \max_{j : (i,j) \in E} p_{ij}$
%\end{claim}

Algorithm~\ref{alg:main} is the rounding algorithm, which we call an {\em iterative rounding} algorithm because it alternates between rounding some variables
and solving the residual problem on the unrounded variables. In every step, the reference to~(\ref{lp:feasibility2}) is with respect to the current bipartite graph $G$ in the algorithm.

\begin{algorithm}[H]
\begin{algorithmic}[1]
\STATE $G \leftarrow (J \cup M;\{(i,j): p_{ij} \leq T\})$ \label{step:pre}
\STATE $T_{i} \leftarrow T \text{ for each }i \in M$
\IF{(\ref{lp:feasibility2}) is infeasible}
	\RETURN {\em no solution}
\ENDIF
\WHILE{$J \neq \emptyset$}
\STATE Find an extreme point solution $\bar{x}$ for~(\ref{lp:feasibility2})
\STATE Delete every edge $(i,j)$ from $E$ such that $\bar{x}_{ij} = 0$
\IF{$\bar{x}_{ij} =1$ for some $(i,j) \in E$}
	\STATE assign $j$ to $i$, reduce $T_{i}$ by $p_{ij}$
	\STATE remove $j$ from $J$  \label{step:int}
\ENDIF
\IF{some $i$ has $\leq 1$ neighbour in $G$ }
	\STATE if $i$ has a neighbour $j$ in $G$, assign $j$ to $i$ and remove $j$ from $G$ \label{step:one}
	\STATE remove $i$ from $G$
\ENDIF
\IF{some $i$ has exactly $2$ neighbours $j,j'$ \textbf{and} $x_{ij} + x_{ij'} \geq 1$}
	\STATE assign $j,j'$ to $i$ \label{step:two}
	\STATE remove $i,j,j'$ from $G$
\ENDIF
\ENDWHILE
\end{algorithmic}
\caption{A Relaxed Decision Procedure via Iterative Rounding}\label{alg:iter}
\label{alg:main}
\end{algorithm}

We will soon prove that in every iteration of the {\bf while} loop that (\ref{lp:feasibility2}) has a feasible solution and that some job or edge is removed from $G$ in each iteration.
If so, then the number of iterations is at most $|E| + |J|$ so it is a polynomial-time algorithm.
The following also holds.
\begin{lemma}
If the algorithm terminates, then assignment has makespan $\leq 2T$.
\end{lemma}
\begin{proof}
Consider the load of a machine $i$ upon termination of the algorithm.
Let $J_{i}^{*}$ be the set of jobs assigned to $i$ over all executions of step \ref{step:int}, and let $J_{i}$ be {\em all} jobs that are assigned to $i$ over all iterations.
Also let $T'_i = \sum_{j \in J_i^*} p_{ij}$.
We first make a few simple observations.

\textbf{Observation 1:} $|J_i - J_{i}^{*}| \leq 2$ because $i$ is removed from $G$ as soon as it is assigned a job $j$ in either step \ref{step:one} or step \ref{step:two}.

\textbf{Observation 2:} $T'_i \leq T$ because we have that $p_{ij} \cdot \overline x_{ij} \leq T_i$ and $\overline x_{ij} = 1$ just when $j \in J_i^*$ is assigned to $i$.
When $j$ is assigned to $i$, we reduced $T_i$ by $p_{ij} = p_{ij} \cdot \overline x_{ij}$, so $T'_i = \sum_{ij} p_{ij} \leq T$.

\textbf{Observation 3:} If $|J_i - J_{i}^{*}| \leq 1$, then $\displaystyle \sum_{j \in J_i} p_{ij} \leq 2 \cdot T$. This is because $J_i = J_i^*$ means the load on machine $i$ is exactly $T'_i \leq T$.
Otherwise, the load is exactly $T'_i + p_{ij'}$ where $j' \in J_i - J_i^*$. Since $(i,j) \in E$, then by step \ref{step:pre} we have $p_{ij'} \leq T$.

The only thing left to show is that if $P_i^* - P_i = \{j,j'\}$ then the load of machine $i$ is at most $2 \cdot T$. 
We bound this as follows, where $T_i$ and $\bar{x}$ refer to the values in Algorithm~\ref{alg:iter} just before $j$ and $j'$ are assigned to $i$ in step \ref{step:two}.
Note that this $T_i$ value is precisely $T - T'_i$ where $T'_i = \sum_{j* \in J_i^*} p_{ij*}$.
\begin{eqnarray*}
\sum_{j'' \in J_i} p_{ij''} & = & T'_i + p_{ij} + p_{ij'} \\
& = & T'_i + (1-\bar{x}_{ij}) \cdot p_{ij} + (1-\bar{x}_{ij'}) \cdot p_{ij'} + \bar{x}_{ij} \cdot p_{ij} + \bar{x}_{ij'} \cdot p_{ij'} \\
& \leq & T'_i + (2-\bar{x}_{ij}-\bar{x}_{ij'}) \cdot T + \bar{x}_{ij} \cdot p_{ij} + \bar{x}_{ij'} \cdot p_{ij'} \\
& \leq & T'_i + T + T_i \\
& = & T'_i + T + (T-T_i) = 2\cdot T
\end{eqnarray*}
Here, the first inequality is because $(i,j), (i,j') \in E$ so $p_{ij}, p_{ij'} \leq	 T$ by step \ref{step:pre}. The second is because $\bar{x}$ is a feasible solution to~(\ref{lp:feasibility2}).

\end{proof}

\begin{lemma}
In every iteration, there is a solution to~(\ref{lp:feasibility2}) and $|E| + |J|$ strictly decreases.
\end{lemma}
\begin{proof}
We first show~(\ref{lp:feasibility2}) has a feasible solution in each iteration. Initially this is true, otherwise Algorithm \ref{alg:iter} would have returned {\em no solution}.
Now consider an iteration that starts with a feasible solution $\bar{x}$ for the LP over the graph $G$.
In the body of this loop, some edges and nodes of $G$ are removed to get a subgraph $G'$. It is easy to verify that in each case, the restriction of $\bar{x}$ to the remaining edges and nodes
remains feasible for the subgraph $G'$, so the next iteration will have a feasible LP solution as well.

Finally, we prove that an edge or job node is always removed from $G$.
So, suppose that at the start of an iteration with the extreme point $\bar{x}$ that $0 < \bar{x}_{ij} < 1$ for each $(i,j) \in E$
and that no $i \in M$ has degree one in $G$ (otherwise some edge or job node will be removed from $G$).
We will show that some $i \in M$ has exactly two neighbours $j,j'$ with $\bar{x}_{ij} + \bar{x}_{ij'} \geq 1$, in which case both $j$ and $j'$ will be removed.

By assumption, every $i \in M$ has ${\rm deg}(i) \geq 2$. For each $j \in J$, because $\sum_{i : (i,j) \in E} \bar{x}_{ij} = 1$ and no $\bar{x}_{ij}$ is exactly 1, then ${\rm deg}(j) \geq 2$ as well.
By the characterization of extreme points, we have that there are at least $|E|$ tight constraints under $\bar{x}$. Since none of these tight are nonnegativity constraints and since there are $|J| + |M|$ other constraints,
then:
\begin{align*}
 |J| + |M| &\geq \# \text{ tight constraints} \\
 & \geq |E| \\
 & = \frac{\sum_{j \in J} {\rm deg}(j) + \sum_{i \in M} {\rm deg}(i)}{2} \\
 & \geq \frac{2|J| + 2|M|}{2} \\
 & = |J|+|M|
\end{align*}
So every inequality must hold with equality. In particular, every node of $G$ has degree exactly 2.
Counting degrees on both sides of $G$, we have $2 \cdot |J| = |E| = 2 \cdot |M|$ so in fact $|J| = |M|$.

We conclude by observing
\[ |M| = |J| = \sum_{j \in J} \sum_{i : (i,j) \in E} \bar{x}_{ij} = \sum_{i \in M} \sum_{j : (i,j) \in E} \bar{x}_{ij}. \]
In particular, there must be some $i \in M$ such that $\sum_{j : (i,j) \in E} \bar{x}_{ij} \geq 1$. This is what we wanted to show: some machine $i$
has degree 2 and $\sum_{j : (i,j) \in E} \bar{x}_{ij} \geq 1$.

\end{proof}

\section{Discussion}
A 2-approximation for this problem was first presented by Lenstra, Shmoys, and Tardos \cite{LST90}.
Currently, the best lower and upper bounds are exactly what are presented in this lecture: the 2-approximation and the NP-hardness of approximating better than 3/2.

Algorithm \ref{alg:iter} can be easily modified to address a more general problem called the \textsc{General Assignment Problem}. The input is much like that for the unrelated machine scheduling problem,
except that the input also includes running time bounds $T_i$ for each machine $i$ and a costs $c_{ij}$ for every potential $j \rightarrow i$ assignment. The goal is to find a minimum $c$-cost assignment
$\phi : J \rightarrow M$ such that no machine $i \in M$ is assigned a running time load of more than $T_i$. A modification of Algorithm \ref{alg:iter} (see~\cite{LRS11} for details) will either find a solution
with cost at most $OPT$ (if there is a feasible solution) where each machine $i$ has running time at most $2 \cdot T_i$. Such an algorithm was initially presented in \cite{ST93}.


It seems we can do more
in an interesting special case. Consider an instance of \textsc{Minimizing Makespan on Unrelated Machines} where each job $j$ can only be processed by some machines, but it has the same
processing time on each of these machines. That is, $p_{ij} \in \{p_j, \infty\}$ for each $j \in J, i \in M$. Svensson shows that for any constant $\epsilon > 0$, we can compute a value $v^*$
such that $OPT \leq v^* \leq (33/17 + \epsilon) \cdot OPT \approx 1.9412 \cdot OPT$ in polynomial time~\cite{S11}.
This is accomplished through rounding a (feasibility) LP relaxation that can be solved with a $(1+\epsilon)$-factor in polynomial time, thus
bounding the ``integrality gap'', but the rounding algorithm itself takes exponential time. Such an algorithm that approximates the optimum solution value without producing a corresponding feasible solution
is sometimes called an {\em estimation algorithm}. It would be interesting to see a polynomial-time approximation algorithm for this case that actually produces
a feasible solution with makespan within some constant factor $c < 2$ of the optimum.

\section*{References}

\begin{itemize}
\item[LRS11] L.~C.~Lau, , R.~Ravi, and M.~Singh. {\em Iterative methods in combinatorial optimization}. Cambridge University Press, 2011.

\item[LST90] J.~K.~Lenstra, D.~B.~Shmoys, and \'{E}.~Tardos. Approximation algorithms for scheduling unrelated parallel machines.
{\em Mathematical Programming}, 46:259--271, 1990.

\item[ST93] D.~B.~Shmoys and \'{E}.~Tardos. An approximation algorithm for the generalized assignment problem.
{\em Mathematical Programming}, 62:461--474, 1993.

\item[S11] O.~Svensson. Santa Claus schedules jobs on unrelated machines. In Proceedings of STOC, 2011.
\end{itemize}

\end{document}
