4 Divide and Conquer Technique

S. Sridhar

Module 10:

Divide and Conquer Technique

This module 10 introduces the concept of popular design technique called divide and conquer technique.

The Learning objectives of this module are as follows:

To understand the concept of Divide and Conquer
To understand applications of Divide and Conquer technique
To know about Merge Sort algorithm
To understand Quicksort Algorithm

What is a divide and Conquer Design Technique?

Divide and conquer is an effective algorithm design technique. This design technique is used to solve variety of problems. In this module, we will discuss about applying divide and conquer technique for sorting problems. In this design paradigm, the problem is divided into subproblems. The subproblems are divided further if necessary. Then the subproblems are solved recursively or iteratively and the results of the subproblems are combined to get the final solution of the given problem.

These are the important components of Divide and Conquer strategy:

Divide: In this stage the given problem is divided into small problems. The smaller problems are similar to the original problem. But these smaller problems have reduced size, i.e., with less number of instances compared to original problem. If the subproblems are big, then the subproblems are divided further. This division process is continued till the obtained subproblems are smaller that can be solved in a straight forward manner.
Conquer: The subproblems can be solved either recursively or non-recursively in a straight forward manner.
Combine: The solutions of the sub-problems can be combined to get the global result of the problems.

Advantages of Divide and Conquer Paradigm

The advantages of divide and conquer approach is that it is perhaps most commonly applied design technique and its application always leads to effective algorithms.
It can be used to solve general problems.
Divide and conquer paradigm is suitable for problems that are inherently parallel in nature.

Disadvantages of Divide and Conquer

The disadvantage of divide and conquer paradigm is that if division process is not carried in a proper manner, the unequal division of problem instances can result in inefficient implementation.

Let us discuss about one of the most popular algorithm that is based on divide and conquer, i.e., merge sort.

Merge Sort

Divide and conquer is the strategy used in merge sort. Merge sort was designed by the popular Hungarian mathematician John van Neumann. The procedure for merge sort is given informally as follows:

Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each.
Conquer: Sort the two subsequences recursively using merge sort in a recursive or non-recursive manner.
Combine: Merge the two sorted subsequences to produce the sorted answer.

Informal algorithm:

Informally merge sort procedure is as follows:

Divide the array A into subarrays L and R of size n/2.
Recursively sort the subarray L gives L sorted subarray
Recursively sort the subarray R gives R sorted subarray
Combine L and R sorted subarrays give final sorted array A

The formal algorithm based on [3] is given as follows:

MergeSort (A, p, r) // sort A[p..r] by divide & conquer

1 if p < r

2 then q ¬ ë(p+r)/2û

3 MergeSort (A, p, q)

4 MergeSort (A, q+1, r)

5 Merge (A, p, q, r) // merges A[p..q] with A[q+1..r]

It can be observed that given array A has p and r as lowest and highest indices. The mid-point p is computed so that the given array is divided into two subarrays. Then the merge sort procedure is called so that the array is recursively divided. Then the procedure uses merge to combine the sorted subarrays, The formal algorithm based on [3] for merging the subarray is given as follows:

Merge(A, p, q, r)

1 n1 ¬ q – p + 1

2 n2 ¬ r – q

3 for i ¬ 1 to n1

4 do L[i] ¬ A[p + i – 1]

5 for j ¬ 1 to n2

6 do R[j] ¬ A[q + j]

7 L[n1+1] ¬ ¥

8 R[n2+1] ¬ ¥

9 i ¬ 1

10 j ¬ 1

11 for k ¬p to r

12 do if L[i] £ R[j]

13 then A[k] ¬ L[i]

14 i ¬ i + 1

15 else A[k] ¬ R[j]

16 j ¬ j + 1

It can be observed that the elements of A is divided into the subarray L and R. Then the elements of L and R are compared and the smaller element is copied to the array A. If the subarray is exhausted, then the remaining elements of the other subarray is copied to array A. ¥ is given as sentinel so that comparison is not done for each and every time for the end of subarray. The following Example 1 illustrates the function of merge sort:

Example 1

Use merge sort and sort the array of numbers {18,26,32,6,43,15,9,1,22,26,19,55,37,43,99,2}

Solution

As said earlier, the first phase of merge sort is to divide the array into two parts using the middle element. The sub-arrays are divided further till one gets an array that cannot be divided further. This division process is shown as below in Fig. 1

Fig 1: Division process

Then, the elements are merged is illustrated for L shown below in Fig. 2.

Fig.2: Division Process of left Subarray

The merge process of left subarray is shown below in Fig. 3

Fig.3: Division Process of left Subarray

Similarly this is repeated for right subarray as well.

Complexity analysis:

If T(n) is the running time T(n) of Merge Sort, then division process for computing the middle takes Q(1), the conquering step , i.e, solving 2 subproblems takes 2T(n/2) and combining step, i.e., merging n elements takes Q(n). In short, the recurrence equation for merge sort is given as follows:

Or in short,

T(n) = Q(1) if n = 1

T(n) = 2T(n/2) + Q(n) if n > 1

Solving this yields, the complexity of merge sort can be derived as :

Þ T(n) = Q(n lg n)

Quicksort

C.A.R. Hoare in 1962 designed Quicksort in 1962. Quicksort uses divide and conquer as a strategy for sorting elements of an array. Merge sort divides the array into two equal parts. But Quick sort unlike merge sort does not divide the array into equal parts. Instead, uses a pivot element to divide the array into two equal parts.

The steps of Quicksort is given below:

Divide step:

Pick any element (pivot) p in S. This is done using a partitioning algorithm. Then, the partitioning element, partition S – {p} into two disjoint groups

S1 = {x Î S – {p} | x <= p}

S2 = {x Î S – {p} | x ³ p}

Conquer step: recursively sort S1 and S2

Combine step: the sorted S1 (by the time returned from recursion), followed by p, followed by the sorted S2 (i.e., nothing extra needs to be done)

The informal Quicksort algorithm is informally given as follows:

1. if left < right:

1.1. Partition a[left…right] such that:

all a[left…p-1] are less than a[p], and

all a[p+1…right] are >= a[p]

1.2. Quicksort a[left…p-1]

1.3. Quicksort a[p+1…right]

2. Combine the subarrays and Terminate

The formal algorithm of quicksort based on [1] is given as follows:

Algorithm quicksort(A, first, last )

%% Input: Unsorted array A [first..last]

%% Output: Sorted array A

Begin

if (first < last) then

v = partition(A,first,last) %% find the pivot element

quicksort([A, first,v-1])

quicksort([A,v+1, last])

end if

end

It can be observed that the important phase of a quicksort algorithm is the partitioning stage where the given array is divided into two parts using a ‘partition’ procedure. While in merge sort, the middle element can be found directly. In quicksort, finding the middle element is not a straight forward process. It is done using partition algorithms.

Partitioning Algorithms

Partitioning algorithms are used to divide the given array into two subarrays. It is complicated process in quicksort compared to the division process of merge sort. There are two partitioning algorithms. One is by Lomuto partitioning algorithm and another by Hoare.

Let us discuss about Lomuto algorithm.

Lomuto Algorithm

Lomuto is a one directional partition algorithm. It scans from left to right and checks for the elements. If the number is less than the pivotal elements, the numbers are swapped. The following example illustrates Lomuto algorithm.

Example 2:

Use Lomuto procedure to partition the following given array:

It can be observed that all the elements of the left of 60 are less than 60 and all the elements on the right hand side is greater than 60.

The formal algorithm based on [1] is given as follows:

ALGORITHM LomutoPartition(A[l..r])

//Partition subarray by Lomuto’s algo using first element as pivot

//Input: A subarray A[l..r] of array A[0..n-1], defined by its //left and right indices l and r (l ≤ r)

//Output: Partition of A[l..r] and the new position of the pivot p <- A[l]

s <- l

for i <- l+1 to r do

if A[i] < p

s <- s+1

swap(A[s], A[i])

swap(A[l], A[s])

return s

Hoare Algorithm

Another useful partition algorithm is called Hoare partition algorithm. This algorithm has two scans. one scan is from left-to-right and another scan is from right-to-left. The left to-right scan (using pointer i) aims to skip the smaller elements compared to the pivot and stop when an element is ³ the pivot. Then right-to left scan (using pointer j) starts with the last element and skips over the elements that are larger than of equal to the pivot element. If i < j, in that case A[i] and A[j] are swapped and the process is continued with the increment of i and decrement of j pointers. If one encounters the situation i > j, then the pivot element is swapped with A[j].

The Hoare partition algorithm is given informally as follows:

Choose pivot element from the array A, generally the first element
Search from left to right looking for elements greater than pivot.
Search from right to left looking for elements smaller than pivot.
When two elements are found, exchange them.
When two elements cross, exchange pivot element such that it is in final place.
Return the pivot element Formally, the Hoare partition algorithm is given as follows:

Algorithm Hoare_partition (A,first,last)

%% Input: Array A with elements 1 to n. First = 1 and last = n %%Output: Sorted array A

Begin

%% First Element is the initial pivot

pivot = A [first]

%% Initialize the pointers i = first+1

j =last

flag = false predicate = true

While (predicate) do

while (i £ j ) and (A[i]

i = i + 1

End while

while (j ³ pivot and j ³ j = j-1

End while

if (j < i)

break

else

A[i] «A[j]

End if

End while

A[first] « A[j]

return j

End

It can be observed that the algorithm initializes two pointers i and j and initial pivot. The pointers are updated based on the conditions that are discussed above as an informal procedure.

The following example illustrates the application of Hoare partition to a given array.

Example 3 : Apply the Hoare portioning algorithm for the following array:

26,33,35,28,19,12,23.

To apply Hoare partition algorithm, the following steps are used:

Step 1: Start with all data in an array, and consider it unsorted

Step 2: Step 1, select a pivot (it is arbitrary), Let it be first element

Step 2, start process of dividing data into LEFT and RIGHT groups. The LEFT group will have elements less than the pivot and the RIGHT group will have elements greater that the pivot.

Step 3: If left element belongs to LEFT group, then left = left + 1. If right index element, belongs to RIGHT, then right = right – 1. Exchange the elements if they belong to the other group.

The final steps are shown below:

Complexity analysis of Quicksort

Quicksort is an effective and popular algorithm and its complexity analysis is given below:

Best Case Analysis: The best case quicksort is a scenario where the partition element is exactly in the middle of the array. The best case quicksort is when the pivot partitions the list evenly. The resulting partitions of a best case are well balanced. Thus, the recurrence equation is given below:

Using the master’s theorem, the complexity of the best case turns out as T (n )Îq (n log n)

It can also be derived as follows:

Worst Case Analysis: In the worst case, it can be observed that the partitions are no longer better than a linear list. This happens because the first element is always the pivot element. Hence, there is no element in the left hand side.

The overall size of the tree is given as

Thus, the worst case complexity of quicksort is q(n2).

In short, one can conclude as part of this module 10 that

Divide and Conquer often leads to a better solution.
Merge sort uses divide and conquer technique and sorts the elements in O(nlogn) time.
Quicksort uses divide and conquer strategy and sorts the elements in o(nlogn) time.
Master Theorem is helpful in solving recurrence equations.

References:

S.Sridhar , Design and Analysis of Algorithms , Oxford University Press, 2014.
A.Levitin, Introduction to the Design and Analysis of Algorithms, Pearson Education, New Delhi, 2012.
T.H. Cormen, C.E. Leiserson, and R.L. Rivest, Introduction to Algorithms, MIT Press, Cambridge, MA 1992.