|Publication number||US8208048 B2|
|Application number||US 12/453,410|
|Publication date||Jun 26, 2012|
|Filing date||May 11, 2009|
|Priority date||Feb 5, 2009|
|Also published as||US20100194902|
|Publication number||12453410, 453410, US 8208048 B2, US 8208048B2, US-B2-8208048, US8208048 B2, US8208048B2|
|Inventors||Huei-Yung Lin, Wei-Zhe Chang|
|Original Assignee||National Chung Cheng University|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (7), Referenced by (15), Classifications (8), Legal Events (2)|
|External Links: USPTO, USPTO Assignment, Espacenet|
1. Field of the Invention
The present invention relates generally to image processing, and more particularly, to a method for high dynamic range imaging.
The objective of high dynamic range imaging is to capture, represent and display the significant amount of luminance variation of the real-world scenes. This problem commonly arises from the limited dynamic range capability of current imaging devices, compared to the much wider perception range of the human visual system. Since the HDRI is aimed to span a large luminance range for vivid image representation and reproduction, it has many applications on, for example, cinematography, photography, computer graphics and visualization, etc.
In addition to the increasing demand of high quality reproduction of real-world scenes, there is always a need for stereoscopic image representation for the binocular human visual system. The objective is to provide us the depth perception ability using stereo image disparities. Thus, there are many advantages and a large application domain available from combining HDRI with stereo vision. For example, the so-called DIBR (depth image based rendering) can be used to generate the content of 3D TV, video games, augmented reality, or used for advanced video surveillance.
2. Description of the Related Art
The most intuitive way of capturing HDR images is to adopt special light detecting sensors capable of recording images with extended intensity range. However, commercially available HDRI systems are usually very expensive and still only accessible to the professional users. For this reason, many researchers have proposed various HDRI techniques using low-cost imaging devices
U.S. Pat. No. 6,864,916 entitled an apparatus and a method for synthesis of high dynamic range imaging disclosed that an image sensor is provided for sequentially capturing multiple images of one scene with different exposures and then combining the multiple images to synthesize a single high dynamic range image.
Taiwan Patent No. I246031 entitled a system and a method for synthesis of images having multiple exposures, based on the above patent, disclosed that an image sensing unit is provided for sensing at least one low dynamic range image, which is the image of the same scene in different exposure time; and an image synthesis unit coupled with the image sensing unit and setting up an exposure history of low dynamic range image for synthesizing the low dynamic image by reference to the exposure history to create a synthetic high dynamic range image. Since each of the low dynamic range images captured by this approach is not done at the same time spot, however, the low dynamic range images are inconsistent with one another to incur afterimage or ghost image in the synthetic high dynamic range image.
The primary objective of the present invention is to provide a method for high dynamic range imaging and disparity computation, by which a plurality of cameras can capture images with different exposures at the same time spot to synthesize a high dynamic range image and derive the disparity map.
The foregoing objective of the present invention is attained by the method including the steps of arranging at least two cameras side by side and meanwhile capturing a plurality of images of one scene with different exposures; adjusting the captured images for the same exposure thereof subject to the response functions of the cameras and then defining a plurality of characteristic spots in the image; combining the characteristic spots corresponding to the images respectively to get a displacement of the corresponding characteristic spot in each of the images and to further get a disparity map; and applying the displacement between the two corresponding characteristic spots in the corresponding images and synthesizing the images to form a synthetic image.
a. Arrange two cameras 11 side by side at the same elevation, the two optical axes of the respective two cameras being parallel to each other, and capture images of one scene with different exposures by the two cameras 11, as shown in
b. Adjust captured images for the same exposure subject to the response functions of the cameras 11 and then define a plurality of characteristic spots 13 in each of the images, as shown in
c. Combine the corresponding characteristic spots 13 to get a displacement between the corresponding two characteristic spots 13 in the corresponding images, further getting a disparity map. Because the two cameras 11 are located at the same elevation, the disparity between the two images is along the horizontal direction.
d. Apply the disparity map and synthesize the images to render a synthetic image shown in
In light of the above steps, the present invention includes the following advantages.
Although the present invention has been described with respect to a specific preferred embodiment thereof, it is no way limited to the details of the illustrated structures but changes and modifications may be made within the scope of the appended claims.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US6864916||Jun 4, 1999||Mar 8, 2005||The Trustees Of Columbia University In The City Of New York||Apparatus and method for high dynamic range imaging using spatially varying exposures|
|US7277118 *||Nov 20, 2001||Oct 2, 2007||Fuji Xerox Co., Ltd.||Method and system for compensating for parallax in multiple camera systems|
|US7683950 *||Apr 26, 2005||Mar 23, 2010||Eastman Kodak Company||Method and apparatus for correcting a channel dependent color aberration in a digital image|
|US20070077056 *||Sep 30, 2005||Apr 5, 2007||Sharp Kabushiki Kaisha||Image shooting apparatus, display apparatus and image shooting and display apparatus|
|US20080024614 *||Jul 25, 2006||Jan 31, 2008||Hsiang-Tsun Li||Mobile device with dual digital camera sensors and methods of using the same|
|US20080143820 *||Dec 13, 2006||Jun 19, 2008||Peterson John W||Method and Apparatus for Layer-Based Panorama Adjustment and Editing|
|TW246031B||Title not available|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US8446481||May 21, 2013||Google Inc.||Interleaved capture for high dynamic range image acquisition and synthesis|
|US8811811||Jul 18, 2013||Aug 19, 2014||Fraunhofer-Gesellscahft zur Foerderung der angewandten Forschung e.V.||Camera system and method for generating high-quality HDR images or videos|
|US8866927||Dec 13, 2012||Oct 21, 2014||Google Inc.||Determining an image capture payload burst structure based on a metering image capture sweep|
|US8866928||Dec 18, 2012||Oct 21, 2014||Google Inc.||Determining exposure times using split paxels|
|US8964060||Aug 8, 2014||Feb 24, 2015||Google Inc.||Determining an image capture payload burst structure based on a metering image capture sweep|
|US8995784||Jan 17, 2013||Mar 31, 2015||Google Inc.||Structure descriptors for image processing|
|US9066017||Mar 25, 2013||Jun 23, 2015||Google Inc.||Viewfinder display based on metering images|
|US9077913||May 24, 2013||Jul 7, 2015||Google Inc.||Simulating high dynamic range imaging with virtual long-exposure images|
|US9087391||Dec 13, 2012||Jul 21, 2015||Google Inc.||Determining an image capture payload burst structure|
|US9100589||Apr 16, 2013||Aug 4, 2015||Google Inc.||Interleaved capture for high dynamic range image acquisition and synthesis|
|US9117134||Mar 19, 2013||Aug 25, 2015||Google Inc.||Image merging with blending|
|US9118841||Dec 24, 2014||Aug 25, 2015||Google Inc.||Determining an image capture payload burst structure based on a metering image capture sweep|
|US9131201||May 24, 2013||Sep 8, 2015||Google Inc.||Color correcting virtual long exposures with true long exposures|
|US9172888||Sep 17, 2014||Oct 27, 2015||Google Inc.||Determining exposure times using split paxels|
|US9247152||Dec 20, 2012||Jan 26, 2016||Google Inc.||Determining image alignment failure|
|U.S. Classification||348/263, 348/239|
|International Classification||H04N9/093, H04N5/262|
|Cooperative Classification||H04N5/2355, H04N5/235|
|European Classification||H04N5/235N, H04N5/235|
|May 11, 2009||AS||Assignment|
Owner name: NATIONAL CHUNG CHENG UNIVERSITY, TAIWAN
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LIN, HUEI-YUNG;CHANG, WEI-ZHE;SIGNING DATES FROM 20081224 TO 20081225;REEL/FRAME:022705/0666
|Dec 28, 2015||FPAY||Fee payment|
Year of fee payment: 4