History is being written in a new way. It is being written by people through the big and the small events in their lives. We write history through personal Web sites, discussion boards, and the legacy of photos, taken at moments that are important to us.
Researchers at Microsoft have been working on a wide range of technologies that will help people write their personal histories through digital photography. To tell any history, it's best to start at the beginning, and move forward.
In the beginning, you buy a digital camera, take a quick look at the manual, throw it to the side, and start pushing buttons. Digital photography has made it possible to take an almost endless number of photos. Some of these photos are good; some get deleted before anyone else sees them. Many of them are almost good, with a little tweaking they'd be just fine.
Adjustable Light
A common problem with digital photography is lighting. Photos turn out either too light or too dark. Since digital cameras allow us to take lots of shots without running out of film, we're willing to throw away a lot of the bad shots. But if you want to take a great picture, it might be nice to be able to control the lighting without purchasing professional lighting equipment.
One of the research projects at Microsoft Research is called Continuous Flash. This technology allows you to take the same picture with flash and without flash and later adjust the balance between light and dark. It's better than contrast filters in photo editing tools, because it considers the reflection characteristics of each object in your picture.
"You can't compensate for having one area underexposed and one area overexposed," said Hugues Hoppe, one of the project researchers. "If an area is underexposed, you can't really get the detail back by increasing brightness, because it wasn't captured in the first place. By having two different images which both have useful information, you can merge them together."
Image Stacks
A similar technology that combines the best of multiple photographs is a project called Image Stacks. Image Stacks aligns multiple images of the same subject, allowing the user to pick and chose the best pieces from each photograph. Researchers Michael Cohen, Steven Drucker and Alex Colburn thought this would come in handy for special events, when you want to get a picture of the entire group that's suitable for framing.
Taking group photographs is difficult, because capturing a single image in which everyone looks good is almost impossible. What usually happens is that in one shot, someone has their eyes closed, but someone else has got the most adorable smile. Check the next shot, everyone has their eyes open, but one person is picking a poppy seed out of their teeth. The third shot, both previous people are behaving, but grandma is yawning, tired of waiting through multiple shots. Which shot do you pick? With Image Stacks, you can easily cut and paste to present everyone's best face. The images are automatically registered into a single composite image.
Print to Digital
Print photos are still around. They're around in shoeboxes. They're shoved under the bed and in the back of closets. Some of us have so many print photos we don't know what to do with them. Neatnik types tediously scan all of their print photos one-by-one, converting them to digital form. But most of us don't want to go to all this trouble.
Cormac Herley, a researcher in the Communications, Collaboration and Signal Processing (CCSP) group, has developed a way to allow people to scan multiple photos at one time. You can put as many photos as will fit on your scanner, and the software will recognize each photo separately. It can 'read' the edges of the photos, even if they're crowded together or tilted. When it converts them to digital photos, it will correct for orientation and position. It's a quick way to get the family memories out of the shoebox and onto your hard drive.
"It's a harder problem than it looks like on the surface. Many scanner makers have tried, but it hasn't worked before," said Herley. "But this really works, it's not just a demo."
Take Out the Red
The Media Computing group at Microsoft Research Asia has developed several cool image editing techniques to help you fix some common problems with any photograph. One of the worst that comes to mind is the glowing red eyes that result from the flash hitting the pupil in just the wrong way. Unless you're way into the vampire look, this just isn't right. The red eye fix, which currently ships in Digital Image Suite and Windows XP Media Center Edition, is one of the best photo retouch features around. All you do is move the 'target' over the red eye, click, and the red is out.
The same group has also developed a quick and effective way to fix the brightness and contrast in a picture. Even if you took a picture that is too dark, you can use the Levels Auto-Fix feature in Digital Image Pro to lighten up your picture.
Organize the Digital Shoebox
The Media Computing group has also made it easy to organize your digital photos using image recognition algorithms. Some people do the same thing they did with the shoebox, and scatter pictures all over their hard drive. Then they can't find the one picture they want to use for their holiday card. The group's algorithms can distinguish between indoor and outdoor shots, shots with people in them and shots without, and city and non-city scenes. Combined with other technologies, such as keyword annotation, it makes it a breeze to find any photo in your collection.
John Platt, a researcher in the CCSP group has developed another way to manage your photos online. His image clustering algorithm helps users find their photos by one of the most prominent markers: events. The software is effective because it doesn't look only at timestamps, which could be misled by a faulty camera clock. Instead, the software looks at photograph order plus color to find pictures taken during one particular event.
"We only compare colors locally in time," said Platt. "So if you have a pumpkin in one shot, and a few months later you wear an orange shirt, later, when you're searching for the day you visited the pumpkin patch, it won't show photographs from when you were wearing the orange shirt."
His algorithm underlies several other technologies designed to help people find their digital photos, including the Microsoft Research Media Browser. The Media Browser, developed by researchers in the Next Media group, takes advantage of the photo recognition research from the Media Computing group, and Platt's algorithms to build a unique visual experience that helps you search for and identify your photos. And it looks darn cool as it works. The interface is an impressive, futuristic presentation of photos that rearrange themselves before your eyes, sliding into place in a typical 2D presentation or a 3D stack.
"The idea behind this is annotation of large collections of photos," said Steven Drucker, the lead researcher on the project. "We know that if you put annotations on photos, that it's much easier to retrieve them. But we also know that it's tedious and difficult to do. We use the advanced techniques that are available, such as face detection and image clustering, to make it easier for you to interact with your photos. We also use a game graphics card for higher visual quality."
Fill It In
Smart Erase is a photo editing tool found in Digital Image Pro and invented by researcher Patrick Perez in Cambridge. The feature allows users to remove objects from a picture. This can come in handy in case you want to remove your ex from the family reunion picture, or you before you lost the 30 pounds.
The algorithm looks at areas of the image to see which patch of texture can be "stolen" to fill in the holes left behind when the unwanted object in the image is removed.
