Using What We Know to Manipulate Images in 3D
Image editing has become quite prolific with companies and individuals using software tools to do everything from removing blemishes to creating practically new scenes. These tools are generally limited to just acting in two dimensions. Researchers at Carnegie Mellon University though have developed software that allows for 3D manipulation of objects in 2D images, by leveraging archives of computer models.
The idea behind the system is to manipulate images based on what we know, instead of just what we see. An object captured in a scene, like a taxi, may only be seen from one angle, but we can recognize that it is a taxi, and from that interpret what it looks like from other angles. The software does essentially the same thing as it semi-automatically aligns 3D models to objects in an image. It also estimates the illumination and the appearance of the hidden portions, so it appears to be seamless, as the object is manipulated.
Of course this technique has limitations, including that there are only so many 3D models for it to draw from, and even as the number increases, it will have to be able to find these new models.
Source: Carnegie Mellon University