Workshop on Commodity-Based Visualization Clusters
Unreliable Transport Protocol for
Commodity-Based OpenGL Distributed Visualization
Samuel Thibault – ENS Lyon
Xavier Cavin – SCI Institute/Inria Lorraine Olivier Festor – Inria Lorraine
Eric Fleury - Inria Rhônes-Alpes
Outline
Motivation
Network interconnect
Distributed OpenGL
Experimentations with Chromium
Conclusion and future works
Workshop on Commodity-Based Visualization Clusters 3
Outline
Motivation
Network interconnect
Distributed OpenGL
Experimentations with Chromium
Conclusion and future works
Motivation
Clusters of commodity computers
Dedicated, expensive network interconnect
– Both transfer time and packets loss are very low – Reliable transport protocol (TCP)
Cheaper network interconnects (… wireless cluster)
– Unreliable transport protocol (UDP) – Degradation of rendering quality
Workshop on Commodity-Based Visualization Clusters 5
Outline
Motivation
Network interconnect
Distributed OpenGL
Experimentations with Chromium
Conclusion and future works
Network interconnect
Transfer Control Protocol (TCP)
– Guaranteed delivery
– Acknowledgment mechanism
User Datagram Protocol (UDP)
– Unreliable transport protocol
– Fast but no delivery or order guarantee
Maximum Transmission Unit (MTU)
– Biggest packet size (no fragmentation) – TCP automatically adapts to the MTU – UDP does not
Workshop on Commodity-Based Visualization Clusters 7
Outline
Motivation
Network interconnect
Distributed OpenGL
Experimentations with Chromium
Conclusion and future works
Distributed OpenGL
Basic brick
Rendering client
OpenGL decoder
OpenGL library OpenGL
encoder OpenGL calls
OpenGL calls OpenGL calls OpenGL calls OpenGL
application
Rendering server
Display Display Network
TCP
UDP
Workshop on Commodity-Based Visualization Clusters 9
Distributed OpenGL
TCP only
Distributed OpenGL
UDP versus TCP
– Remove the acknowledgment mechanism – Some data may not be received
– Or not in the order they were sent
UDP only
– Almost always fatal crash
– Some OpenGL calls shall not be lost: use TCP
– Some can: glBegin/glEnd contents (i.e. glVertex)
Workshop on Commodity-Based Visualization Clusters 11
Distributed OpenGL
Interleaved UDP/TCP
– Synchronization
4 5 5 6 6 6 7
UDP
TCP
Distributed OpenGL
Interleaved UDP/TCP
– Result
Workshop on Commodity-Based Visualization Clusters 13
Distributed OpenGL
Fixing interleaved UDP/TCP
– MTU consideration: no loss example
Expected
1 2
3
4 5
6
7 8
9
Sent
1 2
3
4 5
6
7 8
9
Rendered
Distributed OpenGL
Fixing interleaved UDP/TCP
– MTU consideration: single vertex loss example
Expected
1 2 4
5
6
7 8
9
Received
1 2 4
5
6
7 8
9
Rendered
Workshop on Commodity-Based Visualization Clusters 15
Distributed OpenGL
Fixing interleaved UDP/TCP
– MTU consideration: triplet of vertices loss example
Expected
4 5
6
7 8
9
Received
4 5
6
7 8
9
Rendered
Distributed OpenGL
Fixed interleaved UDP/TCP
– Result
Workshop on Commodity-Based Visualization Clusters 17
Experimentations
Implementation in Chromium
– Framework for OpenGL rendering on clusters of workstations – http://chromium.sourceforge.net
Added features (patches-1-branch of CVS repository)
– IPv6 support
– Interleaved UDP/TCP support
Experimentations
Test applications
– Crocodile: OpenGL viewer
– Rings: Chromium’s test program – Chromium B.S.U.: OpenGL game
Workshop on Commodity-Based Visualization Clusters 19
Experimentations
TCP UDP
Crocodile Rings Chromium B.S.U.
100 Mbps
2.2 fps 2.7 fps
110 fps 110 fps
playable
playable
Experimentations
TCP UDP
Crocodile Rings Chromium B.S.U.
10 Mbps
0.28 fps 0.36 fps
34 fps 39 fps
playable
playable
Workshop on Commodity-Based Visualization Clusters 21
Experimentations
TCP UDP
Crocodile Rings Chromium B.S.U.
Wireless
0. 066 fps 0.37 fps
5.5 fps 8 fps
quite playable
non playable
Conclusion
Interleaved UDP/TCP is adapted:
– If geometry represents most of the traffic
– When rendering quality is not the first interest – When it is animated
– If the network is the bottleneck or has huge loss
It is not:
– When the application parts are closely synchronized – On local high bandwidth networks
Workshop on Commodity-Based Visualization Clusters 23
Future works
Multiple rendering
– UDP IPv6 multicast
– What about TCP packets?
UDP buffering: sendmsg/recvmsg
– MTU constraints – UDP: 1.5 KB
– TCP: 16 KB => less system calls
UDP traffic control
– When sender is too fast for network or receiver – Big losses
Acknowledgments
Support
– VTHD: Very High Broadband (French ministry of industry) – AVTC (DOE VIEWS)
People
– Chromium community – Alain Filbois
– Bruno Levy (Graphite software)
