(page 2 of 2)

Augmented Reality

With augmented reality, it is possible to superimpose computer data and graphics onto the video of the physical model. Then researchers can hold a model in their hands and query the computer for information—asking what a particular residue type is, for instance, or looking at how that residue is conserved across a species.

"The goal is to be able to superimpose any kind of annotation—any information—on top of the physical model," Olson says.

In his office last week, he demonstrated one application of this, which he is developing for a high school in Seattle. He clipped a tiny video camera with a firewire connection onto his shirt and plugged it into his laptop. Then he turned on the computer and held a model of HIV protease in front of the camera. The solid model was captured by the video camera, and after he adjusted the autofocus and launched the software, the model appeared on his computer screen.

This model had a little square marker attached to it that was about the size of a postage stamp and looked like a square bullseye. The bullseye is key to augmented reality because it allows the computer to interpret any given frame of the video image and tell by the shape of the tag what the transformation of the model is based upon the measured distortion of the square marker. Knowing the correct transformation in a frame-by-frame manner allows the computer to track the model in real time as it is moved in front of the camera.

In his demonstration of this application, Olson first entered the coordinates for the atoms in the molecule represented by the physical model he held in his hands. Then he asked the computer to display all the histidine side chains in the molecule. There was only one. "OK," he said, "let's display the phenylalanines, too."

He clicked a few keys, commenting that he is currently working on integrating voice recognition software with this so that he could give simple voice commands to the computer. When he was done, he held up the protein in front of the camera again, and on the computer screen, the video had added graphics representing the His and Phe side chains. The graphics moved as Olson tumbled the protein in his hands.

Then he read in the coordinates for an inhibitor of the HIV protease and he asked the computer to display this. In a few seconds, the computer superimposed the inhibitor on the binding site of the protease. As he turned the model in his hands, the displayed inhibitor turned as well, keeping its correct orientation in the binding site.

The technology is still in development, says Olson, and he is the first to admit that it is primitive. The applications are not fully automated, the video is low resolution, and the display is limited to a computer screen or, at best, a video projector. Someday he envisions integrating augmented reality with a headset display so that different people could look at the same object and each see the particular augmented features they wish to see.

Still, says Olson "It works well enough so that people get a sense of what we're striving for." He feels that this technology as it exists could be a powerful tool for creating tangible interfaces for molecular biology, and he thinks that the application is only going to get better as technology improves.

"You can buy a digital camera today for $100," says Olson. "In 10 years, you might be able to buy an HDTV camera for the same price."

Beauty, Truth, Models, and Everything

Next month, Olson is participating in a forum discussion at the 30th International Conference on Computer Graphics and Interactive Techniques, also known as "Siggraph." This meeting brings together animators and graphics specialists who work in such diverse areas as basic science and entertainment. The panel discussion is titled, "Truth Before Beauty: Guiding Principles for Scientific and Medical Visualization," and like the rest of the convention it brings together experts from different areas to discuss a single subject.

"[The panel] is dealing with the issues of visual representation in the sciences," says Olson. "They invited me to talk about my work in representing the world you can't see—the molecular world."

The molecular world is one that is hard to visualize—even though we see pictures of it on a regular basis. Biology and chemistry have produced thousands and thousands of glimpses of this world in the form of structures, and many of these structures—glossy, full-color, and often quite beautiful—adorn the covers and pages of the top science journals.

The point of the panel is to ask whether these representations are true to what the molecular world looks like or if they sometimes sacrifice truth for the sake of beauty. Olson sees this as a bit of a straw man.

"[The molecular world] doesn't really look like anything," he says.

Since proteins and the other inhabitants of the molecular world are smaller than the wavelength of visible light, they are impossible to see. Structural biologists provide models of molecular structures based on structural data they obtain, and then they usually abstract these models even more—coloring particular residues, displaying only the backbone or particular side chains.

However, proteins have no color as such because color does not exist at that scale. In fact any picture of a protein is not entirely accurate because electron distributions and other dynamic features of proteins do not readily translate into static images. But this is just splitting hairs.

"All representations of the molecular world are models," he says. "None of them are true per se."

The measure of a molecular representation, says Olson, is how well it conveys information. As such, it is often enlightening to show a model that displays less information. If you look at a backbone model of a protein, for instance, you can easily follow the snaking amino acid chain with your eye. Backbone representations have less to do with how the proteins fill space than how they fold up.

Beauty, says Olson, is another question. "Truth and beauty are not really along the same axis."

 

 

1 | 2 |

 

 

 


A tangible strand of DNA. Photo by Jason S. Bardi.