FIGURE 2.20: H.264/AVC video encoder with selectable encoding pa- pa-rameters highlighted
2.6 SUMMARY AND FURTHER READING
This chapter provides some background when transmitting MCP-coded standard-compliant video over error-prone channels. It is important to understand that video can benefit significantly if the transmitter can be sure that the video will be de-livered reliably. Typically, the introduction of error-resilience tools in the video coding layer is very costly in terms of compression efficiency. The overhead is in general much better spent in lower layers of the protocol stack. Nevertheless there exist applications in which errors are inevitable. If the video encoder is not aware
of distortions on the transmission link, this in general leads to dramatic quality degradations due to instantaneous errors as well as spatial–temporal error propa-gation. Whereas the effect of instantaneous errors can be decreased by the use of specific packetization modes, the usually more severe effect of error propagation can be reduced by the application of more frequent intra information, interactive error control, or a combination of both. Preferably, for good overall performance, the selection of error-resilience tools is integrated in rate–distortion-optimized mode selection whereby the channel characteristics should be taken into account in this optimization. In general, standard-compliant decoders such as H.264/AVC can effectively operate even in harsh transmission environments if the encoder is appropriately designed for the transmission conditions and application constraints and the decoder includes some form of appropriate error concealment.
Additional literature on different subjects for error-resilient video transmission is plentiful; some work has already been discussed. In case of detailed interest in different subjects the reader is first of all encouraged to cover the remaining chapters of this book. Furthermore, magazines as well as journals have published special issues that deal exclusively with error-resilient video transmission, for ex-ample, [1,26,49,55,63], which provide a good starting point to dive into deep wa-ters of error-resilient video transmission. Enjoy it!
REFERENCES
[1] J. Apostolopoulos. Reliable video communication over lossy packet networks using multiple state encoding and path diversity. In Proceedings SPIE Visual Communica-tions and Image Processing (VCIP), volume 4310, pages 392–409, San Jose(CA), USA, January 2001.
[2] E. Asbun and E. J. Delp. Real-time error concealment in compressed digital video streams. In Proceedings Picture Coding Symposium, Portland, Oregon, April 1999.
[3] G. Bjøntegaard. Definition of an error concealment model TCON. Doc. ITU-T/SG15/LBC-95-186, ITU-T, Boston, USA, June 1995.
[4] M.-J. Chen, L.-G. Chen, and R.-M. Weng. Error concealment of lost motion vectors with overlapped motion compensation. IEEE Trans. on Circuits Syst. Video Technol., 7(3):560–563, June 1997.
[5] T. P.-C. Chen and T. Chen. Second-generation error concealment for video transport over error prone channels. In Proceedings IEEE International Conference on Image Processing, Rochester(NY), USA, September 2002.
[6] P. A. Chou and Z. Miao. Rate-distortion optimized streaming of packetized media.
IEEE Transactions on Multimedia, 8(2):390–404, April 2006.
[7] G. Cote, S. Shirani, and F. Kossentini. Optimal mode selection and synchronization for robust video communications over error-prone networks. IEEE Journal on Se-lected Areas in Communications, 18(6):952–965, June 2000.
[8] Video adhoc group database for video codec evaluation. Technical Report S4-050789, 3GPP SA4 Video Adhoc Group, September 2005.
[9] A. Eleftheriadis, M. R. Civanlar, and O. Shapiro. Multipoint videoconferencing with scalable video. Journal of Zhejiang University Science, 7(5):696–706, May 2006.
[10] S. Fukunaga, T. Nakai, and H. Inoue. Error resilient video coding by dynamic replac-ing of reference pictures. In Proceedreplac-ings IEEE Globecom, London, United Kreplac-ingdom, November 1996.
[11] M. Ghanbari. Postprocessing of late cells for packet video. IEEE Trans. on Circuits Syst. Video Technol., 6(6):669–678, December 1996.
[12] B. Girod and N. Färber. Feedback-based error control for mobile video transmission.
Proceeding of the IEEE, 97:1707–1723, October 1999.
[13] J. Goshi, A. E. Mohr, R. E. Ladner, E. A. Riskin, and A. F. Lippman. Unequal loss protection for H.263 compressed video. IEEE Trans. on Circuits Syst. Video Technol., 15(3):412–419, March 2005.
[14] B. G. Haskell, A. Puri, and A. N. Netravali. Digital Video: An Introduction to MPEG-2. Chapman & Hall, New York, USA, December 1996.
[15] P. Haskell and D. Messerschmitt. Resynchronization of motion-compensated video affected by ATM cell loss. In Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing, volume 3, pages 545–548, 1992.
[16] S. Hemami and T. Meng. Transform coded image reconstruction exploiting in-terblock correlation. IEEE Transactions on Image Processing, 4(7):1023–1027, July 1995.
[17] ISO/IEC JTC 1. Coding of audio-visual objects, Part 2: Visual. Apr. 1999; Amend-ment 1 (version 2), Feb., 2000; AmendAmend-ment 4 (streaming profile), January 2001.
[18] ITU–T. Video Coding for Low Bit Rate Communication, version 1, November 1995;
version 2, January 1998; version 3, November 2000.
[19] ITU–T and ISO/IEC JTC 1. Advanced Video Coding for Generic Audiovisual Ser-vices, 2003.
[20] B. Jung, B. Jeon, M.-D. Kim, B. Suh, and S.-I. Choi. Selective temporal error con-cealment algorithm for H.264/AVC. In Proceedings IEEE International Conference on Image Processing, Singapore, October 2004.
[21] M. Kalman, P. Ramanathan, and B. Girod. Rate–distortion optimized streaming with multiple deadlines. In Proceedings IEEE International Conference on Image Processing, Barcelona, Spain, September 2003.
[22] M. Karczewicz and R. Kurceren. The SP and SI frames design for H.264/AVC. IEEE Trans. on Circuits Syst. Video Technol., 13(7), July 2003.
[23] C. W. Kim, D. W. Kang, and I. S. Kwang. High-complexity mode decision for error prone environment. Doc. JVT-C101, Joint Video Team (JVT), Fairfax(VA), USA, May 2002.
[24] A. H. Li, S. Kittitornkun, Y. H. Hu, D. S. Park, and J. D. Villasenor. Data partitioning and reversible variable length codes for robust video communications. In Proceedings Data Compression Conference (DCC), Snowbird(UT), USA, March 2000.
[25] J. Liao and J. Villasenor. Adaptive intra update for video coding over noisy chan-nels. In Proceedings IEEE International Conference on Image Processing, volume 3, pages 763–766, October 1996.
[26] A. Luthra, G. J. Sullivan, and T. Wiegand. Special issue on the H.264/AVC video coding standard. IEEE Trans. on Circuits Syst. Video Technol., 13(7), July 2003.
[27] T. Nakai and Y. Tomita. Core experiments on feedback channel operation for H.263+.
Doc. LBC96-308, ITU-T SG15, November 1996.
[28] A. Ortega and K. Ramchandran. Rate–distortion methods in image and video com-pression. IEEE Signal Processing Mag., 15(6):23–50, November 1998.
[29] J. W. Park, J. W. Kim, and S. U. Lee. Dct coefficients recovery-based error conceal-ment technique and its application to the MPEG-2 bit stream error. IEEE Trans. on Circuits Syst. Video Technol., 7(6):845–854, December 1997.
[30] Y. O. Park, C.-S. Kim, and S.-U. Lee. Multi-hypothesis error concealment algorithm for H.26L video. In Proceedings IEEE International Conference on Image Process-ing, pages 465–468, Barcelona, September 2003.
[31] J. Postel. DoD standard transmission control protocol. Request for Comments (stan-dard) 761, Internet Engineering Task Force (IETF), January 1980.
[32] J. Postel. User datagram protocol. Request for Comments (standard) 768, Internet Engineering Task Force (IETF), August 1980.
[33] P. Salama, N. Shroff, E. J. Coyle, and E. J. Delp. Error concealment techniques for encoded video streams. In Proceedings IEEE International Conference on Image Processing, volume 1, pages 9–12, Washington DC, USA, October 1995.
[34] P. Salama, N. Shroff, and E. J. Delp. A Bayesian approach to error concealment in encoded video streams. In Proceedings IEEE International Conference on Image Processing, pages 49–52, Lausanne, Switzerland, September 1996.
[35] P. Salama, N. Shroff, and E. J. Delp. A fast suboptimal approach to error concealment in encoded video streams. In Proceedings IEEE International Conference on Image Processing, pages 101–104, Santa Barbara(CA), USA, October 1997.
[36] P. Salama, N. B. Shroff, and E. J. Delp. Error concealment in encoded video. In Image Recovery Techniques for Image Compression Applications. Kluwer Academic Press, Dordrecht, The Netherlands, 1998.
[37] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson. RTP: A transport protocol for real-time applications. Request for Comments (standard) 3550, Internet Engineer-ing Task Force (IETF), July 2003.
[38] C. E. Shannon. A Mathematical Theory of Communications. Bell Systems Technol-ogy Journal, pages 379–423, 623–656, 1948.
[39] S. Shirani, F. Kossentini, and R. Ward. A concealment method for video communi-cations in an error-prone environment. IEEE Journal on Selected Areas in Communi-cations, 18(6):1122–1128, June 2000.
[40] T. Socolofsky and C. Kale. A TCP/IP tutorial. Request for Comments (standard) 1180, Internet Engineering Task Force (IETF), January 1991.
[41] E. Steinbach, N. Färber, and B. Girod. Standard compatible extension of H.263 for ro-bust video transmission in mobile environments. IEEE Trans. on Circuits Syst. Video Technol., 7(6):872–881, December 1997.
[42] T. Stockhammer and M. Bystrom. H.264/AVC data partitioning for mobile video communication. In Proceedings IEEE International Conference on Image Process-ing, pages 545–548, Singapore, October 2004.
[43] T. Stockhammer, M. M. Hannuksela, and T. Wiegand. H.264/AVC in wireless envi-ronments. IEEE Trans. on Circuits Syst. Video Technol., 13(7):657–673, July 2003.
[44] T. Stockhammer, T. Wiegand, and D. Kontopodis. Rate-distortion optimization for JVT/H.26L coding in packet loss environment. In Proceedings International Packet Video Workshop, Pittsburgh(PA), USA, April 2002.
[45] G. J. Sullivan and T. Wiegand. Video compression – from concepts to the H.264/AVC standard. Proceeding of the IEEE, 93(1):18–31, January 2005.
[46] G. J. Sullivan and T. Wiegand. Rate-distortion optimization for video compression.
IEEE Signal Processing Mag., 15(6):74–90, November 1998.
[47] H. Sun and W. Kwok. Concealment of damaged block transform coded images using projections onto convex sets. IEEE Transactions on Image Processing, 4:470–477, April 1995.
[48] H. Sun and J. Zedepski. Adaptive error concealment algorithm for MPEG com-pressed video. In Proceedings SPIE Visual Communications and Image Processing (VCIP), pages 814–824, Boston(MA), USA, November 1992.
[49] R. Talluri. Error-resilient video coding in the ISO MPEG-4 standard. IEEE Commu-nications Magazine, 36(6):112–119, June 1998.
[50] Y. Tomita, T. Kimura, and T. Ichikawa. Error resilient modified inter-frame coding system for limited reference picture memories. In Proceedings Picture Coding Sym-posium, Berlin, Germany, September 1997.
[51] D. S. Turaga and T. Chen. Model-based error concealment for wireless video. IEEE Trans. on Circuits Syst. Video Technol., 12(6):483–495, June 2002.
[52] V. Varsa, M. M. Hannuksela, and Y.-K. Wang. Non-normative error concealment al-gorithms. Doc. VCEG-N62, ITU-T SG16/Q6 Video Coding Experts Group (VCEG), Santa Barbara(CA), USA, September 2001.
[53] W. Wada. Selective recovery of video packet losses using error concealment. IEEE Journal on Selected Areas in Communications, 7:807–814, June 1989.
[54] B. W. Wah, X. Su, and D. Lin. A survey of error concealment schemes for real-time audio and video transmissions over the internet. In IEEE International Symposium on Multimedia Software Engineering, pages 17–24, Taipei, Taiwan, December 2000.
[55] Y. Wang, S. Wenger, J. Wen, and A. G. Katsaggelos. Error resilient video coding techniques. IEEE Signal Processing Mag., 17(4):61–82, July 2000.
[56] Y. Wang and Q. Zhu. Error control and concealment for video communication: A re-view. Proceeding of the IEEE, 86:974–997, May 1998.
[57] Y.-K. Wang, M. M. Hannuksela, V. Varsa, A. Hourunranta, and M. Gabbouj. The er-ror concealment feature in the H.26L test model. In Proceedings IEEE International Conference on Image Processing, volume 2, pages 729–732, Rochester(NY), USA, September 2002.
[58] S. Wenger. Common conditions for wireline, low delay IP/UDP/RTP packet loss resilient testing. Doc. VCEG-N79, ITU-T SG16/Q6 Video Coding Experts Group (VCEG), Santa Barbara(CA), USA, September 2001.
[59] S. Wenger, M. M. Hannuksela, T. Stockhammer, M. Westerlund, and D. Singer. RTP payload format for H.264 video. Request for Comments (standard) 3984, Internet Engineering Task Force (IETF), February 2005.
[60] S. Wenger, G. Knorr, J. Ott, and F. Kossentini. Error resilience support in H.263+.
IEEE Trans. on Circuits Syst. Video Technol., 8:867–877, November 1998.
[61] T. Wiegand, N. Färber, K. Stuhlmller, and B. Girod. Error-resilient video transmission using long-term memory motion-compensated prediction. IEEE Journal on Selected Areas in Communications, 18(6):1050–1062, June 2000.
[62] T. Wiegand and B. Girod. Lagrangian multiplier selection in hybrid video coder con-trol. In Proceedings IEEE International Conference on Image Processing, Thessa-loniki, Greece, October 2001.
[63] T. Wiegand and B. Girod. Multi-Frame Motion-Compensated Prediction for Video Transmission. KL, KLADR, 2001.
[64] T. Wiegand, X. Zhang, and B. Girod. Long-term memory motion-compensated pre-diction. IEEE Trans. on Circuits Syst. Video Technol., 9(1):70–84, February 1999.
[65] H. Yang and K. Rose. Source–channel prediction in error-resilient video coding. In Proceedings IEEE ICME, Baltimore(MD), USA, July 2003.
[66] W. Zeng and B. Liu. Geometric-structure-based error concealment with novel ap-plications in block-based low-bit-rate coding. IEEE Trans. on Circuits Syst. Video Technol., 9(4):648–665, June 1999.
[67] R. Zhang, S. L. Regunthan, and K. Rose. Video coding with optimal inter/intra-mode switching for packet loss resilience. IEEE Journal on Selected Areas in Communica-tions, 18(6):966–976, June 2000.
[68] Q. F. Zhu and L. Kerofsky. Joint source coding, transport processing, and error con-cealment for H.323-based packet video. In Proceedings SPIE Visual Communications and Image Processing (VCIP), volume 3653, pages 52–62, San Jose(CA), USA, Jan-uary 1999.
[69] W. Zia and F. Shafait. Reduced complexity techniques for long-term memory mo-tion compensated predicmo-tion in hybrid video coding. In Proceedings Picture Coding Symposium, Beijing, China, April 2006.