Software transforms digital photos into old-fashioned paintings

While today’s most cutting-edge digital imaging technology strives for high-res and high focus, Stephen Brooks is interested in distorting photos and deliberately making them unfocused. His system can produce a unique piece of artwork that looks hand-made, whether by oil painting, watercolors, ink drawing, or a mixture of many media types.

Brooks is a computer scientist at Dalhousie University, and also has a background in traditional drawing. The method he recently developed can transform digital photos taken with an ordinary camera into mixed-media paintings and portraits, making him a pioneer in a subject area that is largely unexplored. While the system requires very little input from the user, the results can be tweaked if desired, and produce realistic compositions of “nonphotorealistic renderings” (NPR).

“Mixed media compositions incorporate two or more traditional visual media such as oil, watercolor, tempera, ink, or pencil,” Brooks explained. “By mixing materials, the artist is able to explore new channels of artistic expression or rejuvenate existing ones.”

As Brooks explains, mixed media is often used by artists when a single medium is insufficient to fully develop an artistic idea. From ancient Egyptians to Rembrandt and Picasso, and even contemporary cartoonists such as Dave McKean, mixed media has been used to give variety to work and emphasize certain details.

To give the appearance of mixed media to photos, Brooks’ system analyzes the input image and computes a measure of detail at each pixel. The areas of the image with different levels of detail are separated into distinct regions, and then each region is processed independently with a different NPR filter chosen by the user. Examples of filters can be “paint daubs,” “soft glow,” “crystallize,” “ink outlines,” “cartoon,” etc. Choosing the filters is the only step that requires user input.

Finally, the different areas of the image are smoothly fused. Here, the user has the option of determining the gradient level of blended areas, but Brooks notes that all the samples he presents in his research are created automatically by the system. The system performs the blending process in separate stages to allow the user to see the intermediate processing. Brooks explains that this step-by-step process resembles how an artist produces a mixed media work in stages.

One of the most challenging types of images to transform is facial portraits, as people are highly sensitive to the treatment of faces. Using Brooks’ standard technique, the facial areas around the cheeks and forehead are perceived as low detail areas, while the eyes and mouth have more detail, which produces undesirable results. Brooks approached this special case by developing systems for skin filtering, face point detection, and face point clustering to create a more satisfactory mixed media portrait.

“The most advanced component of my system is the module that handles portraits,” Brooks said. “Automatic face detection and processing is quite tricky if one wants to make the system robust. I use a combination of skin detection, cascaded weak detectors, and Gaussian mixture model clustering.”

Brooks is currently exploring more advanced features and improvements, including integrating 2.5-dimensional physical materials such as felts and cardboards, as well as the possibility of adapting the technique to video.

“We have recently begun collaborating with the film school at the Nova Scotia College of Art and Design,” he said. “We are looking at developing a new kind of digital film production system which could involve extending mixed media processing to video. So, it may be released as part of a larger film processing package.”

Citation: Brooks, Stephen. “Mixed-Media Painting and Portraiture.” IEEE Transactions on Visualization and Computer Graphics, vol. 13, no. 5, September/October 2007.

Related stories:

DNA analysis of anthrax fingered suspect: reports
DNA analysis of anthrax in victims' bodies led investigators to the government scientist who killed himself just before he was charged in the deadly 2001 anthrax attacks, US media reported Monday.
US anthrax suspect commits suicide (Update)
A US government scientist has committed suicide just as he was about to be charged in the deadly 2001 anthrax attacks that sparked widespread panic in the United States, US media reported Friday.
Ethanol byproduct produces green results
Commercial flower and plant growers know all too well that invasive, ubiquitous weeds cause trouble by lowering the value and deterring healthy growth of potted ornamental plants. To control weeds, many commercial nursery owners resort to the expensive practice of paying workers to hand-weed containers. Some growers use herbicides, but efficacy of herbicides is questionable on the wide range of plant species produced in nurseries, and many herbicides are not registered for use in greenhouses.
Walk your way to a healthier lifestyle
Need a boost to get off the couch? A new study shows that a variety of interventions designed to promote walking can effectively motivate individuals to initiate walking behaviors. The results of the review are published in the July issue of Medicine & Science in Sports and Exercise.
Applying genomic medicine into clinical practice for chronic diseases still in the early stages
A large gap exists between what knowledge is available about genomic medicine and incorporating it into clinical practice for assessing the risk of and treating common chronic diseases, such as cardiovascular disease, diabetes mellitus, and cancer, according to a systematic review in the March 19 issue of JAMA, a theme issue on Genetics and Genomics.
Honey bee invaders exploit the genetic resources of their predecessors
Like any species that aspires to rule the world, the honey bee, Apis mellifera, invades new territories in repeated assaults. A new study demonstrates that when these honey bees arrive in a place that has already been invaded, the newcomers benefit from the genetic endowment of their predecessors.
Butterfly fish 'may face extinction'
A beautiful black, white and yellow butterflyfish, much admired by eco-tourists, divers and aquarium keepers alike, may be at risk of extinction, scientists have warned.
New techniques create butanol -- biofuel superior to ethanol
A team of researchers headed by an environmental engineer at Washington University in St. Louis is plying new techniques to produce a biofuel superior to ethanol.

News discussion:

Boring! in Technology news