March242020 JanHubiˇcka AlgorithmsanddatastructuresILecture8:selfbalancingtrees

(1)

Recall AVL-tree insert AVL-tree delete (a,b)-trees

Algorithms and datastructures I Lecture 8: self balancing trees

Jan Hubiˇcka

Department of Applied Mathematics Charles University

Prague

March 24 2020

(2)

Set datastructure

We would like to represent aset(or a dictionary) of some elements from anuniversum.

We expect that elements of the universum in set can be assigned and compared inO(1) INSERT(v): Insertvto the set

DELETE(v): Deletevfrom the set FIND(v): Findvin the set MIN: Return minimum MAX: Return maximum SUCC(v): Find successor PRED(v): Find predecessor Basic implementations

INSERT DELETE FIND MIN/MAX SUCC/PRED

Linked list O(n)orO(1) O(n)orO(1) O(n) O(n) O(n)

Array O(n)orO(1) O(n)orO(1) O(n) O(n) O(n)

Sorted array O(n) O(n) O(logn) O(1) O(logn)orO(1)

(3)

Binary search trees

Definition (Binary tree) Binary treeis:

1. a rooted tree where

2. every vertex has at most 2 sons and

3. we where distinguishleftandrightson of every vertex Notation: for a vertexvin a binary tree we denote by

l(v)andr(v)the left and right son ofv, p(v)the parent ofv.

T(v)the subtree rooted inv,

L(v)andR(v)the subtree rooted in left and right son ofv, h(v)the height ofT(v).

Definition (Binary search tree)

Binary search treeis a binary tree where every vertexvhas uniquekeyk(v)and for every vertexvit holds: 1. ∀_x∈L(v):k(x)<k(v)and

2. ∀_y∈R(v):k(y)>k(v).

(4)

Binary search trees

3. we where distinguishleftandrightson of every vertex

2. ∀_y∈R(v):k(y)>k(v).

(5)

Binary search trees

2. ∀_y∈R(v):k(y)>k(v).

(6)

Binary search trees

Binary search treeis a binary tree where every vertexvhas uniquekeyk(v)and for every vertexvit holds:

1. ∀_x∈L(v₎:k(x)<k(v)and 2. ∀_y∈R(v₎:k(y)>k(v).

(7)

AVL-trees (1962)

Georgy Adelson-Velsky Evgenii Landis

Definition (AVL tree)

Binary search tree isheight balanced(orAVL-tree) if

∀_v:

h(l(v))−h(r(v)) ≤1.

Lemma

Every AVL-tree withnvertices has heightΘ(logn)

(8)

AVL-trees (1962)

Georgy Adelson-Velsky Evgenii Landis

Definition (AVL tree)

Binary search tree isheight balanced(orAVL-tree) if

∀_v:

h(l(v))−h(r(v)) ≤1.

Lemma

Every AVL-tree withnvertices has heightΘ(logn)

(9)

Insert operation

Remember for every vertex asignδ(v) =h(r(v))−h(l(v)) Insert(v,x)

1. Insert element to a binary search tree 2. Re-balance the tree

Given vertexxwe need to to solve the situation where its sonsincrease height by 1. Assume thatyis a left son (for right son the situation is symmetric). Consider three cases:

1. δ(x) = +(right subtree is higher):

Putδ(x) =0. We are finished: result is AVL-tree again.

2. δ(x) =0(both subtrees are having same height):

Putδ(x) = +and recursively rebalance inp(x)(subtree ofxjust got higher). 3. δ(x) =−(left subtree is higher):

Subtree ofxis not height balanced anymore. We need to userotationsto fix it. Look atδ(y)and consider individual cases:

(10)

Insert operation

(11)

Insert operation

(12)

Insert operation

Putδ(x) = +and recursively rebalance inp(x)(subtree ofxjust got higher).

(13)

Insert operation

3. δ(x) =−(left subtree is higher):

(14)

Insert operation

Subtree ofxis not height balanced anymore. We need to userotationsto fix it.

Look atδ(y)and consider individual cases:

(15)

Rebalancing for δ(x) = − and δ(y ) = −

x −2 y −

C h A

h+1 B h h+2

y x A

h+1 B h

C h

h+1 0

0

(16)

Rebalancing for δ(x) = − and δ(y ) = −

x −2 y −

C h A

h+1 B h h+2

y x A

h+1 B h

C h

h+1 0

0

(17)

Rebalancing for δ(x) = − and δ(y ) = +

x −2 y

z ^D h

A

h B

h

⁻

C h

⁻

h+2

h+1

z

y x

A h

B h

⁻

C h

⁻

D h

h+1 h+1

+

(18)

Rebalancing for δ(x) = − and δ(y ) = +

x −2 y

z ^D h

A

h B

h

⁻

C h

⁻

h+2

h+1

z

y x

A h

B h

⁻

C h

⁻

D h

h+1 h+1

+

(19)

Rebalancing for δ(x) = − and δ(y ) = 0

This case never happens. We only propagate up from vertex with sign+or−.

Lemma

OperationINSERTon AVL-tree can be implemented inΘ(logn)time.

Proof.

We know that the height of AVL-tree isΘ(logn). INSERTto binary search tree is done inΘ(logn).

Re-balancing may recurse to a father, but number of changes is again limited by the height of tree. See, for example,https://gist.github.com/Twoody/de8d079842e0dd20cf20d870c73168af. Good advice: when implementing AVL-tree write also a verifier that all invariants are maintained correctly.

(20)

Rebalancing for δ(x) = − and δ(y ) = 0

Lemma

Proof.

We know that the height of AVL-tree isΘ(logn).

INSERTto binary search tree is done inΘ(logn).

Re-balancing may recurse to a father, but number of changes is again limited by the height of tree.

Good advice: when implementing AVL-tree write also a verifier that all invariants are maintained correctly.

(21)

Rebalancing for δ(x) = − and δ(y ) = 0

Lemma

Proof.

We know that the height of AVL-tree isΘ(logn).

INSERTto binary search tree is done inΘ(logn).

Re-balancing may recurse to a father, but number of changes is again limited by the height of tree.

See, for example,https://gist.github.com/Twoody/de8d079842e0dd20cf20d870c73168af.

Good advice: when implementing AVL-tree write also a verifier that all invariants are maintained correctly.

(22)

Delete operation

Delete(v,x)

1. Delete element from a binary search tree 2. Re-balance the tree

son (for right son the situation is symmetric). Consider three cases: 1. δ(x) =−(left subtree is higher):

Putδ(x) =0and recursively rebalance inp(x)(subtree ofxjust decreased height)

2. δ(x) =0(both subtrees are having same heigh):

Putδ(x) = +. We are finished.

Subtree ofxis not height balanced anymore. We need to use rotations to fix it. Look atδ(y)and consider individual cases

(23)

Delete operation

Delete(v,x)

Given vertexxwe need to to solve the situation where its sonsdecreasesheight by 1. Assume thatyis a left son (for right son the situation is symmetric). Consider three cases:

(24)

Delete operation

Delete(v,x)

(25)

Delete operation

Delete(v,x)

(26)

Delete operation

Delete(v,x)

(27)

Delete operation

Delete(v,x)

(28)

Delete operation

Delete(v,x)

Subtree ofxis not height balanced anymore. We need to use rotations to fix it.

Look atδ(y)and consider individual cases

(29)

Rebalancing for δ(x) = + and δ(y ) = +

x y A

h B

h C h+1

h+2

y x

C h+1 A

h B h h+1

+2 +

0

(30)

Rebalancing for δ(x) = + and δ(y ) = +

x y A

h B

h C h+1

h+2

y x

C h+1 A

h B h h+1

+2 +

0

(31)

Rebalancing for δ(x) = + and δ(y ) = 0

x y A

h B

h+1 C h+1

h+2

− y x

C A h+1

h B h+1 h+2

+2

0 +

(32)

Rebalancing for δ(x) = + and δ(y ) = −

x

y − A z

h D

h B

h

⁻

C h

⁻

h+2 h+1

z

x y

A h

B h

⁻

C h

⁻

D h

h+1 h+1

+2 0

(33)

Rebalancing for δ(x) = + and δ(y ) = −

x

y − A z

h D

h B

h

⁻

C h

⁻

h+2 h+1

z

x y

A h

B h

⁻

C h

⁻

D h

h+1 h+1

+2 0

(34)

Theorem

OperationsINSERT,DELETE,FIND,MIN,MAX,SUCC,PRED, on AVL-trees can all be implemented inΘ(logn)time.

Theorem

Every datastructure for set which only use comparison on the elements of the universum must implementFINDin Ω(logn)time.

Proof.

Assume that set containsnelements. OperationFIND(x) hasn+1possible answers. Every comparison has only 3 possible answers.

(35)

Theorem

ForINSERT,DELETE,FINDthis is best possible.

Theorem

Proof.

(36)

Theorem

(37)

Theorem

Proof.

(38)

(a, b)-trees (Bayer, McCreight)

AVL-trees do few compares, but use a lot of memory.

Can we combine both?

Rudolf Bayer Edward M. McCreight

Definition (Generalized search tree)

Generalised search treeis a rooted tree with specified order of sons and two types od vertices:

1. Internalvertices contains non-zero number of keys. If internal vertex has keysx₁<· · ·<xnthen it hask+1sonss₀, . . . ,s_k. Keys separate values in sons, so:

T(s0)<x1<T(s1)<x2<· · ·<xk−1<T(sk−1)<xk <T(sk) 2. Externalvertices contain no keys and are leaf.

Definition ((a,b)-tree)

(a,b)-treefor a givena≥2,b≥2a−1is a generalised search tree such that: 1. Root has 2 tobsons.

2. Other internal vertices haveatobsons. 3. All external vertices are in the level.

(39)

(a, b)-trees (Bayer, McCreight)

Sorted arrays use less memory, but theINSERTandDELETEoperations are slow.

(40)

(a, b)-trees (Bayer, McCreight)

(41)

(a, b)-trees (Bayer, McCreight)

(42)

(a, b)-trees (Bayer, McCreight)

(a,b)-treefor a givena≥2,b≥2a−1is a generalised search tree such that:

1. Root has 2 tobsons.

2. Other internal vertices haveatobsons.

3. All external vertices are in the level.

(43)

Height of (a, b)-trees

(a,b)-treefor givena≥2,b≥2a−1is generalised search tree such that 1. Root has 2 tobsons.

2. Other internal vertices haveatobsons 3. All external vertices are in the same height

Lemma

Every(a,b)-tree withnkeys has depthΘ(logn).

Proof.

Analyse the minimum number of vertices(a,b)-tree of heighthcan have. Level0has one vertex (root) with at least2keys. Levellhas at leastatimes as many keys as levell−1. This grows exponentially fast.

Analogously we can analyse maximum number of vertices.

(44)

Height of (a, b)-trees

(a,b)-treefor givena≥2,b≥2a−1is generalised search tree such that 1. Root has 2 tobsons.

2. Other internal vertices haveatobsons 3. All external vertices are in the same height

Lemma

Every(a,b)-tree withnkeys has depthΘ(logn).

Proof.

Analyse the minimum number of vertices(a,b)-tree of heighthcan have. Level0has one vertex (root) with at least2keys. Levellhas at leastatimes as many keys as levell−1. This grows exponentially fast.

Analogously we can analyse maximum number of vertices.

(45)

Find and insert to (a, b)-tree

Find(v,x)

Find operation can be implemented similarly to one on binary search tree.

1. Ifvis external vertex return∅.

2. Look into keys invifxis found then return it.

3. If it is not found chose right subtree to recurse into.

Insert(v,x)

Letube the last internal vertex visited by Find(v,x). 1. Ifucontainsxreturn.

2. Otherwise addxintouand insert new external vertex 3. Ifuhas more thanbsons, split it.

(46)

Find and insert to (a, b)-tree

Find(v,x)

Find operation can be implemented similarly to one on binary search tree.

1. Ifvis external vertex return∅.

2. Look into keys invifxis found then return it.

3. If it is not found chose right subtree to recurse into.

Insert(v,x)

Letube the last internal vertex visited by Find(v,x).

1. Ifucontainsxreturn.

2. Otherwise addxintouand insert new external vertex 3. Ifuhas more thanbsons, split it.

(47)

Splitting of a vertex

Preventive splitting

Useful simplification: Ifb≥2athen we can preventively split every vertex withbsons during the descent to the tree.

(48)

Splitting of a vertex

Preventive splitting

Useful simplification: Ifb≥2athen we can preventively split every vertex withbsons during the descent to the tree.

(49)

Delete from a (a, b)-tree

Delete(v,x) 1. u←Find(v,x) 2. Ifu=∅return

3. Ifuis not in the lowest level of the tree: 4. s←Succ(u,x)

5. Replacexinubys

6. u←vertex which contains keys. 7. Removexfromu.

8. Ifuhas fewer thanasuns see if we can borrow a key from left or right sibling. 9. If not mergeuwith sibling.

Theorem

OperationsINSERT,DELETE,FIND,MIN,MAX,SUCCandPREDon(a,b)-tree runs inΘ(logn)time.

(50)

Delete from a (a, b)-tree

3. Ifuis not in the lowest level of the tree:

4. s←Succ(u,x)

5. Replacexinubys

6. u←vertex which contains keys.

Theorem

(51)

Delete from a (a, b)-tree

4. s←Succ(u,x)

5. Replacexinubys

7. Removexfromu.

Theorem

(52)

Delete from a (a, b)-tree

4. s←Succ(u,x)

5. Replacexinubys

7. Removexfromu.

8. Ifuhas fewer thanasuns see if we can borrow a key from left or right sibling.

Theorem

(53)

Delete from a (a, b)-tree

4. s←Succ(u,x)

5. Replacexinubys

7. Removexfromu.

8. Ifuhas fewer thanasuns see if we can borrow a key from left or right sibling.

9. If not mergeuwith sibling.

Theorem

March242020 JanHubiˇcka AlgorithmsanddatastructuresILecture8:selfbalancingtrees

Algorithms and datastructures I Lecture 8: self balancing trees

Set datastructure

Binary search trees

Binary search trees

Binary search trees

Binary search trees

AVL-trees (1962)

AVL-trees (1962)

Insert operation

Insert operation

Insert operation

Insert operation

Insert operation

Insert operation

Rebalancing for δ(x) = − and δ(y ) = −

x −2 y −

C h A

h+1 B h h+2

y x A

h+1 B h

C h

h+1 0

0

Rebalancing for δ(x) = − and δ(y ) = −

x −2 y −

C h A

h+1 B h h+2

y x A

h+1 B h

C h

h+1 0

0

Rebalancing for δ(x) = − and δ(y ) = +

x −2 y

z D h

A

h B

h

C h

h+2

h+1

z

y x

A h

B h

C h

D h

h+1 h+1

+

Rebalancing for δ(x) = − and δ(y ) = +

x −2 y

z D h

A

h B

h

C h

h+2

h+1

z

y x

A h

B h

C h

D h

h+1 h+1

+

Rebalancing for δ(x) = − and δ(y ) = 0

Rebalancing for δ(x) = − and δ(y ) = 0

Rebalancing for δ(x) = − and δ(y ) = 0

Delete operation

Delete operation

Delete operation

Delete operation

Delete operation

Delete operation

Delete operation

Rebalancing for δ(x) = + and δ(y ) = +

x y A

h B

z ^D h

z ^D h