On-the-Fly Coding for Real-Time Applications

O

pen

A

rchive

T

oulouse

A

rchive

O

uverte (

OATAO

)

OATAO is an open access repository that collects the work of Toulouse researchers

and makes it freely available over the web where possible.

This is an author-deposited version published in:

http://oatao.univ-toulouse.fr/

Eprints ID: 3424

To link to this article: DOI:

10.1016/S1296-2139(00)01075-7

URL:

http://dx.doi.org/10.1016/S1296-2139(00)01075-7

To cite this version: TOURNOUX, Pierre-Ugo, BOUABDALLAH, Amine, LACAN,

Jérôme, LOCHIN, Emmanuel. On-the-Fly Coding for Real-Time Applications. In :

Proceedings of the 2009 ACM Multimedia Conference and co-located workshops,

October 19 - 24, 2009, Beijing, China. New York, N.Y. : ACM Press, 2009. pp. 889-892.

ISBN 978-1-60558-608-3

Any correspondence concerning this service should be sent to the repository administrator:

On-the-Fly Coding for Real-Time Applications

Pierre-Ugo Tournoux, Amine Bouabdallah, Jérôme Lacan and Emmanuel Lochin

{ptournoux, amine.bouabdallah, jerome.lacan, emmanuel.lochin}@isae.fr

CNRS; LAAS

Université de Toulouse; ISAE

ABSTRACT

Althoughironi ally itdoesnotoer any real-time guaran-tee, Internet is a popular solution to support multimedia time- onstrainedappli ations(e.g. VoIP,Video Conferen -ing, ...). Following this trend, this paper fo uses on the performan e of these appli ations by studying the benet of using a novel reliability on ept whi h aims at signi- antlyimprovingtheperforman eofthesetime onstrained appli ationsoverlossybest-eortnetworks. Thisreliability me hanism emerged from several re ent works from both network and oding theories. Its prin iple is to integrate feedba ksinanon-they odings hemeinordertooptimize thetrade-o"pa ketde odingdelay"vs"throughput". We presenttherstevaluationsofthisme hanismforVoIPand video- onferen ingappli ationsforvariouserasure hannels. Comparedto lassi blo k-basederasure odes,the results show signi ant gains interms of quality observed by the userforbothappli ations.

Keywords

Reliability,Erasure ode,VoIP,Video- onferen ing

General Terms

Algorithms,Design

1.

INTRODUCTION

The su ess of multimediaappli ations is well established overtheInternet. However,thenetworkdoesnotbringany delay guarantees and an IP onne tion path an be om-posedof severallink te hnologies thatmight in lude long-delay links. However, among these appli ations, some of themhavestrongreal-time onstraints. Thisistypi allythe aseforvideo- onferen ingandVoIP.Forexample,VoIP ap-pli ationsare hara terizedbyamaximum"mouth-to-ear" thresholdwhereadelaylowerthan200ms[2℄ismandatory toget agood ommuni ationquality. Asamatteroffa t, thedelay onstraintsare omplextosatisfy.

Thisproblembe omesextremelyhardinthe ase ofpa ket losses due to ongestion or transmissions impairment. To date,twokindsofreliabilityme hanismsare ommonlyused to re overpa ketlosses: ForwardError Corre tion(FEC) andAutomati RepeatreQuests(ARQ).Bothareusedwhen thereisnostrongdelay onstraintsastheysigni antly in- reasethetransmissiondelay. Indeed,retransmissionsused by ARQ imply at least one additional Round Trip Time (RTT)whi hisuna eptable inmost ases; whileFEC in- reasesthedelaybe auseofthedataen odingwhi hisdone onaper-blo kbasis. ThismeansthattheFECde oderhas to re eiveenough pa kets froma blo k to re over thelost data.

Theen odingblo ksize anberedu ed,howeverforaxed odingrate,thelowertheblo ksize,thelowerthede oding performan e. It meansthat atequal orre tion apability, ade reaseoftheblo ksize(and onsequently,ofthepa ket de odingdelay)mustbefollowedbyade reaseofthe oding rate(and onsequently,oftheappli ationthroughput). Someproposalstoimprovetherelationshipsbetweenpa ket de odingdelay, blo klength andthroughput are givenin[8℄ and [6℄. The ombination of FEC and feedba ks, alled Hybrid-ARQ, an also improve the trade-o between reli-ability and throughput[9℄, however, it doesnot solve the problemofthepa ketde odingdelay,espe iallywhen trans-missiondelaysare onsidered.

Re ently,severalworksonerasureandnetwork oding pro-posedto in ludethefeedba ksinthe odingpro essin or-der to redu e the pa ket de oding delay. Sundararajan et al. [10℄haveproposedanon-the-ynetwork odings heme in ludingfeedba ks. Thiss hemeallowstoredu ethe aver-agede odingdelayinthe aseofseveralre eivers. Intheir evaluation,they hoose to negle ttransmission delays and thedelays onsideredresultfromthelossesobservedby dif-ferent re eivers. Oneofthemain ontributionofthis work is the on ept of"seen pa ket",whi hallows there eivers toa knowledgethedegreesoffreedomofthelinearsystem orrespondingtothere eivedpa kets. Themainadvantage ofthiss hemeistheoptimizationofthebuerso upan y. Independently,La anandLo hin[3℄haveproposedanother on-the-y odingsystemusingfeedba ksinthe aseof point-to-point ommuni ationswithhightransmissiondelays. Ba-si ally,theprin ipleistoaddrepairpa ketsgeneratedasa linear ombination of all the sour edata pa ketssent but

toenablefull-reliabilityindelaytolerantnetworksandmore pre iselyindeepspa enetworkswhereana knowledgment pathmightnotexistor wheredelay mightpreventtheuse ofARQs hemes. Theanalysis ofthisme hanismprovided in[7℄showspromisingresults.

Therst ontributionof our paperis the identi ationof thesedierent on epts as a solutionto improve real-time multimediaappli ations(Se tion2). We hooseto usethe me hanism proposed in [3℄ and integrate the on ept of "seenpa kets"[10℄whi hallowstoredu eslightlythe me h-anism's omplexity. These ond ontributionistheanalysis ofthe performan es obtainedbythis me hanismwith two ommonly-usedreal-timeappli ations overvarious erasure hannel hara teristi s.

Wetesttheperforman eofourproposalwithaVoIPtra (Se tion3)andevaluatetheresultswiththeMeanOpinion S ore value of the E-Model[2℄. We show that thequality obtainedissigni antlyhigher omparedto lassi erasure odings hemes. Then,weevaluatethisme hanismwiththe morestringentdelayrequirementsofvideo- onferen ing ap-pli ations(Se tion4)andshowthatthequalityofthevideo obtainedintermsofPSNRoutperformsthoseobservedwith lassi me hanisms.

2.

A RELIABILITY SCHEME FOR

REAL-TIME APPLICATIONS

Themainprin ipleofthiserasure odings hemeisto gen-erate repair pa kets from a variable-sizeen oding window ontaining the sour e pa kets

P

i

, i > 0

, providedby the sour eappli ationandnotyeta knowledgedbythere eiver. Wedenote

R

(i..j)

=

P

j

u=i

α

(i,j)

u

.P

u

arepair pa ketbuiltas alinear ombinationofsour epa ketsrangingfrom

i

to

j

. The oe ients

α

(i,j)

u

arerandomly hosenintheniteeld

F

q

and are transmittedinthe headerofthe repair pa ket. A repair pa ketis generated ea h

k

sour epa kets, where

k

,whi hdeterminesthe oding rate(equalto

1/(k + 1)

)is determineda ording to the pa ketloss rate. The sour e pa ketsaresentunmodied(systemati ode).

Atthere eiverside,thede oder "subtra ts"ea hre eived sour e pa ket to all re eived repair pa kets inwhi h it is involved. Itfollows thattheset ofresulting repairpa kets is a set of linear ombinations of the lost sour e pa kets. Thede oding onsistssimplyinsolvingthislinearsystem. This anbedone assoonasthenumberofre eivedrepair pa kets (theequations)is at leastequal to the numberof lostsour epa kets(theunknowns).

The en oding window is emptied by the feedba ks of the de oderwhi ha knowledgethere eptionorthede odingof thesour epa kets.Thefeedba k ontainsapa ketsequen e number below whi h all the previous pa kets are known. Thepa ketswithasequen enumberhighertothisnumber arein ludedinasele tivea knowledgmentve tor.This a -knowledgments hemeallowstoredu ethenumberofsour e pa ketsinvolvedinthe en oding/de oding ata given time whileitmaintainsthefull-reliabilityproperty.

ex hange inFig. 1. Inthis example, during the data

ex-P3 P4

P3 P4

P3 P4

P3 P4

P3 P4

P3 P4

P3 P4

R(9,10)

P10

P9

R(1..8)

P8

P7

R(1..6)

P6

P5

R(1..4)

P4

P3

R(1,2)

P2

P1

P3

P2

Missing Pkts

Redundancy Pkts

R(1,2)

R(1..6)

R(1..6) R(1..8)

R(9,10)

Figure1: A simple dataex hange

hange,pa ket

P

2

islost. However,therepairpa ket

R

(1,2)

, su essfully arrived,allows to rebuild

P

2

. Weassume that there eiversendsana knowledgmentpa kettoinformthe sender that it an ompute the next repair pa kets from pa ket

P

3

(i.e. this pa ket a knowledges pa kets

P

1

and

P

2

). Then,this a knowledgmentis lost. Howeverthisloss does not ompromise the following transmissions and the sendersimply ontinuesto omputerepairpa ketsfrom

P

1

. After this, we see that

P

3

, P

4

and

R

(1..4)

pa ketsare lost. Noneofthesepa ketsneedtoberetransmittedastheyare rebuiltthankstothepa ketsre eivedfrom

P

5

to

R

(1..8)

. In-deed,there eiver anrebuild

P

3

, P

4

byrstlysubtra ting all the re eived sour e pa kets from the repair pa kets in ordertoobtain

(R

′

(1..6)

, R

′

(1..8)

)

whi harelinear ombina-tions of

P

3

and

P

4

. Bysolving this linearsystem,

P

3

and

P

4

anbere overed.

In this reliability s heme the purpose of the a knowledg-mentsistoredu ethenumberofsour epa ketshandledby arepairpa ketasillustratedinFig. 1,where,afterre eiving thea knowledgment,therepairpa ket

R

(9,10)

startsfrom

9

andnotfrom

1

. Theseenpa ket on eptintrodu edin[10℄ allowstoremovepa ketsfromtheen odingwindowevenif theyare not yetde oded. Asanexampleafter the

re ep-tionof

R(1..6)

there eiverwas allowedto a knowledge

P

3

as aseenpa ket,redu ing slightlythesize oftheen oding window at the sender side. The re eiver was also able to a knowledge

P

5

and

P

6

withaSACKblo kredu ing more signi antlythesizeof thewindow. Finally,we annoti e thatthelossofa knowledgmentsdoesnot ompromise relia-bility(inFig. 1,ifnofeedba kisre eived,thesendersimply sends

R

(1..10)

), even ifit in reases the odingwindow and onsequentlyitimpa ts omplexity.

Tosummarizethisshortintrodu tion,thiss hemepermitsa full-reliabilityevenifsomesour edatapa kets,repair pa k-etsora knowledgmentsarelost. Moreinterestinglyfor the ontextofreal-timeappli ations,thede odingdoesnot de-pend onthe feedba ks and thus the loss re overydelay is

3.

VOIP WITH TETRYS

2.5

3

3.5

4

4.5

0.05

0.1

0.15

0.2

MOS

Packet Loss Rate

Tetrys

FEC 3 4

FEC 6 8

FEC 9 12

FEC 12 16

(a)BernouilliChannel

2.5

3

3.5

4

4.5

0.05

0.1

0.15

0.2

MOS

Packet Loss Rate

Tetrys

FEC 3 4

FEC 6 8

FEC 9 12

FEC 12 16

(b)GilbertElliott withaverageburstsize=3

Figure2: MOSfor the G711 ode as afun tion of thePLR.Theen odingratioisxedto3/4andthe FEC performan es are shown with dierent blo k size.

This se tion presentsan evaluation of the performan e of TetrysfortheVoIPappli ation. Sin ethedelaysdueto re-transmissionsareinmost asesuna eptable,wedonot on-siderARQandwe ompareTetryswithblo k-basedFEC. Weshowasubsetofthesimulationsettingsthat wetested with notably various Pa ket Loss Rate (PLR) generated withBernoulli(i.i.d. losses) 2(a)andGilbert-Elliott(bursty losses) 2(b) hannels. Ourresultsareissuedfroma stand-alone simulator whi h models the link layer losses events and the proto ol's behavior. Sin ethe blo k size impa ts ontheFEC performan esfor a given oding rate,we also show the result of FEC for a oding rate of

3/4

and for blo k of sizes4, 8, 12and 16. To obtainthe same oding rate,Tetrysgenerates arepairpa ketevery3sour e pa k-ets. Therepair pa kets(bothFECand Tetrys)are sentas soon as possible (depending onthe link layer apabilities) aftertheir omputation. Thetransmission delayisxedto

100

ms and a sour e pa ket is generated every

10

ms. At there eiverside,wemeasurethe observedpa ketlossrate andthedelay(in ludingthetransmissionandthede oding delays). These two valuesare used to estimate the Mean OpinionS ore(MOS)obtainedthankstotheE-model[2℄as aqualitymetri s.

Figures 2(a) and 2(b) show the MOS fa tor evolution as afun tion of the PLR for the G711 ode . The two sub-guresdierfromthetypeofunderlying hannel;Fig 2(a) and2(b)showtheMOSforrespe tivelyaBernoulli hannel andaGilbertElliott hannelwithanaverageburstsizeof3. OntheBernoulli hannel,we anseethatfora20%pa ket loss rate, Tetrys obtains a stable and signi antly better MOSvalue omparedtoFECme hanisms. Tetrysrebuilds thelostpa ketundera eptabledelaywhilethepa ketsnot rebuiltbyFECimpa titsMOSfa tor. This omparisonof theme hanismsremainstrue for thebursty hannel. One analsonoti ethatuser'sexperien eremains"good"(MOS

≥

4.0

) for FECup to PLR equalto 12.5%and 5% for re-spe tively Bernoulli and Gilbert-Elliott hannels, whereas Tetrys obtains "good" quality for PLR up to respe tively 17.5% and 14%. A given setof parameters is thenusable underalargerbreathingspa ethanFEC.Italsopermitsto in reasethe odingrate,allowingthentoredu etheloadof thenetworkduetotherepairpa kets.

We anseethatwhentheMOSforTetrysbe omessmaller thanthe MOS ofFEC, the qualityof FEC is between3.4 and 2.8 whi h is onsidered as "poor" by the users. This meansthat whentheperforman e ofTetrysde reases, the oneofFECisalreadynota eptable.

4.

VIDEO-CONFERENCING WITH TETRYS

We hoose topresent theperforman e ofTetrysina time- onstrained and dynami ontext. Indeed, amongst real-time appli ations, video- onferen ing requires signi antly lessthanonese ondend-to-enddelayandisknownto per-form ideally when the delay does not ex eed 100 ms (see [11℄). Video appli ation is also hara terized by its nat-ural variable bit rate (VBR), even if re ent video oders (as H.264) an generate onstant bit rate(CBR) on aver-age. With regardstoTetrys, thesetwolast hara teristi s of the video onferen e appli ations impose, a ording to the instantaneous videobit-rate, avariableupperlimiton theusable(notobsolete)numberofpa ketsat there eiver side. Consequently,atthe senderside,it indire tlydenes avariableupperlimit onthe odingwindowsize, (beyond thatupperlimit,the odedpa ketswouldbelateanywayat there eiverandwill notbe onsidered). WhileFEC-based s heme annotbenetmu hfromthispropertysin eit op-erates onx-blo k lengthwhateverthe upperlimit onthe usable pa kets on the re eiver side, Tetrys automati ally expands its oding window whendata losses o ur, hen e improvingthe probability ofre overingfromlosses. Sin e, the higherthe sour ebit rate, the bigger the usable od-ing window(fora xend-to-enddelay),Tetryshen e pro-vides a stronger prote tion to pa kets that belong to high bitratedata.Forvideo odeddata,sin eI-frameshavethe higherbit-rate, they will benetmost from the adaptabil-ity ofTetrys. Furthermore,it isalso knownthat I-frames, whenlost, anhaveaworseimpa tontheenduserquality omparedtoP-framesorB-frames. Itfollowsthat,in addi-tiontoitsex ellent performan eon onstant-bitrateows, Tetrys intrinsi ally provides a kind of unequal error pro-te tionlikeother(lessexible)erasure s hemesspe i ally designed [1℄. Consequently,we expe tsigni ant improve-mentofthevideoqualityin onversationalvideoappli ation whenusingTetrys omparedto lassi FECs hemes. Contrary to error resilient oding s heme like [4℄, Tetrys aimstobeusedatthetransportlayerlevelanditisnot spe- i toany ode orappli ation. Thatisthereasonwhy,as forVoIP,we ompareTetryswithvariableFECs hemes un-dervariableloss rateinBernoulliandGilbert-Elliott han-nels. Asavideo ode ,weusethelastITU-T'svideo ode Re ommendationH.264and theJM15.1 H.264/AVC soft-ware. WeranoursimulationonForemansequen e,inCIF size, witha frameskip of onepi ture resulting ina frame rateof

15

fps. OneI-frameisinsertedevery

14

P-framesand noB-frames were insertedbe ause ofthe extradelaythey generate. Theaveragebit-rateisabout

384

kbpsatthe

out-ChannelType PLR Tetrys 34 68 912 1216 Bernouilli 5% 35.67 33.47 34.6 32.32 30.91 10% 34.38 26.75 31.39 29.19 27.46 15% 31.77 20.26 24.69 25.58 23.34 20% 26.02 15.32 18.70 20.05 20.23 GilbertEliot 5% 32.66 27.48 30.17 30.85 31.30 withaverage 10% 28.75 20.88 24.05 24.89 25.86 burst 15% 24.11 16.70 19.48 20.85 21.39 size=3 20% 18.81 15.03 16.42 17.87 18.12

Table 1: PSNR ofFECand Tetrys

0

1

2

3

4

5

6

7

8

9

10

0

200

400

600

800

1000

1200

Packet rate (pk/s)

0

1

2

3

4

5

6

7

8

9

10

10

20

30

40

50

PSNR (dB)

Time (s)

Average PSNR (dB): Tetrys= 31.62 FEC= 26.66

Tetrys

fec k=6 n =8

Figure3: Pa ketrateand instantaneous PSNRofa livevideowithPLRof15%ontheBernoulli hannel

putofthevideo oderandthe odedstreamispa kedinto pa ketsof

500

byteslength. Themaximaltolerable end-to-enddelay issetto

200

ms,hen eallpa ketsre eivedafter this due time are dropped. A total of

150

oded frames orresponding

to

10s of video is used. Inorder to obtain representativeresults,ea htestisrepeated

20

times ( orre-spondingto

3000

framesand

200

sofvideo). Thissetupis derivedfrom the ommon testing onditions mentionedin [11℄. ForevaluatingthevideoweusetheEvalvidframework des ribed in [5℄ where the video quality is measured with thePeak SignaltoNoise Ratio (PSNR)metri wherehigh s oresmeansbetterquality.

AsillustratedthroughexperimentsgiveninTable1,Tetrys learly outperforms all the tested FEC s hemes inall s e-narii. Fortheaverageperforman esontheBernoulli han-nel,Tetrysa hievesagainof

6.19

dBataloss rateof

15

% overthebestFECs heme(FEC912)forthiss enario. Fur-thermore,thedropinPSNRforTetrysdoesnotex eed

4

dB whenthepa ketloss ratein reasesfrom

5

%to

15

%;hen e ensuringthe average PSNRis always above

30

dB. When fullreliabilityisimpossiblebe auseofhightime- onstraints, Tetrysallowsgra efuldegradationofthevideoquality. On theGilbert-Elliott Channel,theresultskeepthe same ten-den yeven if thegains are less important whenthe burst lengthin reases(

2.72

dB foraverageburstlengthof

3

and FEC1216). Thesesigni ant gainsarea hievedthanksto theexibilityofTetrys. Thisparti ularityofTetrysisbetter illustratedinFigure3whosebottomgraphgivesasnapshot of

10

se onds of the evolution of the instantaneous PSNR

andthetopgraphrepresentstheinstantaneouspa ketrate ofthevideodatawherepeakpointsrepresenttheo urren e ofI-frames. We an learlyseeonthisguresthatTetrys re-trieves

9

I-framesoutof

10

,whereasFECs hemessu eeded

5

times. Statisti spro essedonthewholesequen eshowed thatTetrysretrievesonaverage

84

%I-frameswhereasFEC s hemes su eeded only

60

%of the time over a Bernoulli hannel and respe tively

73

% and

40

% over a Gilbert El-liott hannel.

Andlastbutnotleast,forvideodata,ithappenedthatthe highbitratedatahasthemostimportantee tontheend uservideoquality. Consequently,itturnsoutthatTetrysis providinge ient transparent unequalprote tion to video data. It is worth noting that Tetrys doesnot requireany extrainformationex hanges(aboutdatatypesandsizesand so forth) from the appli ation while most of the existing unequalprote tions hemesdo.

5.

CONCLUSION

These results are en ouraging and give a proof ofthe via-bilityoftheon-the-y odingforreal-timeappli ation. We gave two examples of real-time appli ation to whi h full-reliability is notmandatory. Future work will in lude the extensionof theserstperforman e evaluationsonvarious kinds of appli ations. Theanalysis of the behavior of the me hanism,notablythe hoi eoftheredundan yratio a - ording to the ontext and appli ation requirements will be deeplystudied. Thesimulationsshowed that the od-ing/de oding omplexitiesarelow omparedtoblo k odes, butthismustbe onrmedbytheoreti alanalysis.

6.

REFERENCES

[1℄ A.BouabdallahandJ.La an.Dependan y-aware unequalerasureprote tion odes.PV

15

th

Int.Pa ket VideoWorkshop,7:2733,May2006.

[2℄ ITU.T-REC-G.107-200503-I,TheE-Model,a

omputationalmodelforuseintransmissionplanning. [3℄ J.La anandE.Lo hin.Rethinkingreliabilityfor

long-delaynetworks.InInt.WorkshoponSatelliteand Spa e Com.,(IWSSC),pages9094,O t.2008. [4℄ M. Kim,H.Oh,N.Dutt,A.Ni olau,and

N.Venkatasubramanian.Pbpair: anenergy-e ient error-resilienten odingusingprobabilitybasedpower aware intrarefresh.SIGMOBILEMob.Comput. Commun.Rev.,10(3):5869,2006.

[5℄ J.Klaue,B.Rathke,andA.Wolisz.Evalvid-A frameworkfor videotransmissionandquality evaluation.InPerforman e Tools,pages255272, 2003.

[6℄ J.KorhonenandP.Frossard.Flexibleforwarderror orre tion odeswithappli ationtopartialmedia datare overy.SignalPro essing: Image

Communi ation,24:229242, 2009.

[7℄ J.La anandE.Lo hin.On-the-y odingtoenable full reliabilitywithoutretransmission.

orre tion odeswithlowde odingdelay.IEEE Transa tionsonInformationTheory,O t.2004. [9℄ A.Sahai.Whydoblo klengthanddelaybehave

dierentlyiffeedba kispresent? IEEETransa tions onInformationTheory,54(5):18601886, May2008. [10℄ J.K.Sundararajan,D.Shah,andM.Medard.ARQ fornetwork oding.IEEEInt.Symp.onInformation Theory,pages16511655, July2008.

[11℄ S.Wenger.H.264/AVCoverIP.IEEETrans.Cir uits Syst.VideoTe hn,13(7):645656,2003.