In OpenGL one needs to specify the viewing volume. This is done in the way indi-cated in Figure 4.22. Note that there is no view plane as such and that the z-axis does not point in the direction one is looking but in the opposite direction. The view volume is specified by the far clipping plane z = -f and a rectangle [R,r]¥[b,t] in the near clip-ping plane z = -n.
The steps in OpenGL are
(1) Convert to camera coordinates.
(2) Map the homogeneous camera coordinates to homogeneous clip space, but this time one maps into the cube [-1,1]¥[-1,1]¥[-1,1]. The homogeneous matrix M that does this is defined by the equation
The matrix M is obtained in the same manner as was MhcamÆhclipin Section 4.5. The call to the function glFrustum (R,r,b,t,n,f) in OpenGL generates the matrix nM.
(3) Project to normalized device coordinates in Euclidean space (division by w).
(4) Transform to the viewport.
4.12 Reconstruction
Ignoring clipping, which we shall in this section, by using homogeneous coordinates the mathematics in our discussion of the graphics pipeline basically reduced to an equation of the form
(4.13) where M was a 4 ¥3 matrix, aŒR4, and bŒR3. The given quantities were the matrix M, computed from the given camera, and a point in the world that determined a. We then used equation (4.13) to compute band the corresponding point in the view plane.
Our goal here is to give a brief look at basic aspects of two types of inverse problems.
For additional details see [PenP86]. For a much more thorough and mathematical discussion of this section’s topic see [FauL01].
The Object Reconstruction Problem. Can one determine the point in the world knowing one or more points in the view plane to which it projected with respect to a given camera or cameras?
The Camera Calibration Problem. Can one determine the world-to-view-plane transformation if we know some world points and where they get mapped in the view plane?
Engineers have long used two-dimensional drawings of orthogonal projections of three-dimensional objects to describe these objects. The human brain is quite adept at doing this but the mathematics behind this or the more general problem of recon-structing objects from two-dimensional projections using arbitrary projective trans-formations is not at all easy. Lots of work has been done to come up with efficient solutions, even in what might seem like the simpler case of orthographic views. See, for example, [ShiS98]. Given three orthographic views of a point (x,y,z), say a front, side, and top view, one would get six constraints on the three values x, y, and z. Such overconstrained systems, where the values themselves might not be totally accurate in practice, are typical in reconstruction problems and the best that one can hope for is a best approximation to the answer.
Before describing solutions to our two reconstruction problems, we need to address a complication related to homogeneous coordinates. If we consider projec-tive space as equivalence classes [x] of real tuples x, then mathematically we are really dealing with a map
(4.14) Equation (4.13) had simply replaced equation (4.14) with an equation of representa-tivesa, M, and bforp, T, and q, respectively. The representatives are only unique up to scalar multiple. If we are given pand T and want to determine q, then we are free to choose any representatives for p and T. The problems in this section, however, involve solving for pgiven T and bor solving for T given pandq. In these cases, we
T
cannot assume that the representatives in equation (4.13) all have a certain form. We must allow for the scalar multiple in the choice of representatives at least to some degree. Fortunately, however, the equations can be rewritten in a more convenient form that eliminates any explicit reference to such scalar multiples. It turns out that we can always concentrate the scalar multiple in the “b” vector of equation (4.13).
Therefore, rather than choosing the usual representative of the form b=(b1,b2,1) for [b], we can allow for scalar multiples by expressing the representative in the form
(4.15) Leta=(a1,a2,a3,a4) and M =(mij). Let mj= (m1j,m2j,m3j,m4j), j =1,2,3, be the column vectors of M. Equation (4.13) now becomes
It follows that c =a•m3. Substituting for c and moving everything to the left, equa-tion (4.13) can be replaced by the equaequa-tions
(4.16a) (4.16b) It is this form of equation (4.13) that will be used in computations below. They have a scalar multiple for bbuilt into them.
After these preliminaries, we proceed to a solution for the first of our two recon-struction problems. The object reconrecon-struction problem is basically a question of whether equation (4.13) determines aif M and bare known. Obviously, a single point bis not enough because that would only determine a ray from the camera and provide no depth information. If we assume that we know the projection of a point with respect to two cameras, then we shall get two equations
(4.17a) (4.17b) At this point we run into the scalar multiple problem for homogeneous coordinates discussed above. In the present case we may assume that M¢ =(mij¢) and M≤ =(mij≤) are two fixed predetermined representatives for our projections and that we are looking for a normalized tuple a= (a1,a2,a3,1) as long as we allow a scalar multiple ambiguity in b¢ = (c¢b1¢,c¢b2¢,c¢) and b≤ =(c≤b1≤,c≤b2≤,c≤). Expressing equations (4.17) in the form (4.16) leads, after some rewriting, to the matrix equation
(4.18)
and
This gives four equations in three unknowns. Such an overdetermined system does not have a solution in general; however, if we can ensure that the matrix A has rank three, then there is a least squares approximation solution
(4.19) using the generalized matrix inverse A+(see Theorem 1.11.6 in [AgoM05]).
Next, consider the camera calibration problem. Mathematically, the problem is to compute M if equation (4.13) holds for known points ai andbi, i =1, 2, . . . , k. This time around, we cannot normalize the aiand shall assume that ai=(ai1,ai2,ai3,ai4) and bi =(cibi1,cibi2,ci). It is convenient to rewrite equations (4.16) in the form
(4.20a) (4.20b) We leave it as an exercise to show that equations (4.20) can be written in matrix form as
where
and
This overdetermined homogeneous system in twelve unknowns mijwill again have a least squares approximation solution that can be found with the aid of a generalized inverse provided that nis not zero.