|Publication number||US6962450 B2|
|Application number||US 10/661,000|
|Publication date||Nov 8, 2005|
|Filing date||Sep 10, 2003|
|Priority date||Sep 10, 2003|
|Also published as||EP1515265A2, EP1515265A3, US20050053405|
|Publication number||10661000, 661000, US 6962450 B2, US 6962450B2, US-B2-6962450, US6962450 B2, US6962450B2|
|Inventors||Patrick Brouhon, Ira Goldstein|
|Original Assignee||Hewlett-Packard Development Company L.P.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (18), Non-Patent Citations (8), Referenced by (33), Classifications (10), Legal Events (5)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This application is related to the following Patent Applications: U.S. patent application Ser. No. 10/661001 filed Sep. 10, 2003, entitled “Printing Digital Documents”; U.S. patent application Ser No. 10/660324 filed Sep. 10, 2003, entitled “Location Patterns And Methods And Apparatus For Generating Such Patterns”; U.S. patent application Ser. No. 10/660324 filed Sep. 10, 2003, also entitled “Location Patterns And Methods And Apparatus For Generating Such Patterns”; British Patent Application No. GB20030021164 filed Sep. 10, 2003, entitled “Methods, apparatus and software for printing location pattern” and, British Patent Application No. GB20030021168 filed Sep. 10, 2003, entitled “Printing of documents with position identification pattern”.
The present invention relates to methods and apparatus for generating position identifying pattern, which can be detected by a suitable detection system. The pattern may be applied to a product such as a document, which may be a form, label or note pad, or any other form of product suitable for such marking, such as a packaging product.
It is known to use documents having such position identification pattern in combination with a pen having an imaging system, such as an infra red sensitive camera, within it, which is arranged to image a small area of the page close to the pen nib. The pen includes a processor having image processing capabilities and a memory and is triggered by a force sensor in the nib to record images from the camera as the pen is moved across the document. From these images the pen can determine the position of any marks made on the document by the pen. The pen markings can be stored directly as graphic images, which can then be stored and displayed in combination with other markings on the document. In some applications the simple recognition that a mark has been made by the pen on a predefined area of the document can be recorded, and this information used in any suitable way. This allows, for example, forms with check boxes on to be provided and the marking of the check boxes with the pen detected. In further applications the pen markings are analysed using character recognition tools and stored digitally as text. Systems using this technology are available from Anoto AB and described on their website www.Anoto.com.
In order to allow documents to be produced easily with the position identifying pattern on them, it is desirable for the pattern to be suitable for printing on the types of printer that are readily available to a large number of users, such as an ink jet, laser jet or LEP printer. These are digital printers and typically have a resolution of 300, 600 or 1200 dots per inch, and the accuracy with which each dot can be located is variable. Also such printers are generally either monochrome, or, if they are colour printers, have only a small number of ink colours. Therefore, if it is desired to print position coding pattern on a part of a product which has human visible content on it as well, it can be a problem to ensure that the position identifying pattern can be distinguished from the content by the reading device, and that the content remains clearly visible to the human eye, and distinguishable over the content.
According to a first as aspect of the invention there is provided a method of generating an image comprising a position identifying pattern and a content feature, the method comprising the steps of: generating the pattern and the content feature each as a plurality of graphical elements, and superimposing the content feature and the pattern, wherein the content elements are smaller than the pattern elements in at least one dimension. This can enable the pattern elements within the superimposed area to be machine read, for example by a digital pen.
The step of generating the content feature may comprise the steps of: defining the content feature, determining whether the content feature is to be superimposed on the pattern and, if it is, converting the content feature so that it comprises said content elements. This ensures that substantially any content feature can be printed with the pattern. Clearly some initial content features will be modified more than others in the conversion process to enable them to be distinguished from the pattern. Content features which are already formed from a number of graphical elements may simply require changes to the size or spacing of those elements. Content features which are initially solid colour, for example solid black, will need to be broken down into separate graphical content elements.
The method may comprise, before the converting step, determining whether the content feature already comprises said content elements and, only if it does not, performing the converting step. This allows features which are already in a form which can be superimposed on the pattern, without preventing the pattern from being read, to be printed in their original form without undergoing any further modification.
The content marks may be smaller in two dimensions, which may be orthogonal dimensions, than the pattern marks, and may each be smaller in area than the pattern marks.
The difference in size between the pattern elements and the content elements, which is required to enable the pattern to be machine read, will depend on the details of the reading device. If the reading device is arranged to recognize marks in a predetermined range of sizes as being pattern elements, then the content elements need to be of a size that is well outside that range to ensure that the reading device does not erroneously identify the content elements as pattern elements. For example the content elements may be no bigger than half as big, in said one dimension, as the pattern elements. Where the content elements comprise discrete dots, they may be, on average, no bigger than a third, or even a quarter, of the area of the pattern elements.
When applied to a product the pattern elements may each be formed from a plurality of dots or pixels merged together to form a substantially solid mark, and the content elements may each be formed from at least one dot or pixel. This is how the product can be printed on a printer, such as an inkjet, laser jet or LEP printer. Such printers apply ink or toner in a large number of discrete areas, or pixels, which are the smallest areas that the printer can mark individually. The content elements may therefore each comprise a single pixel, thereby being as small as the printer can make them. Alternatively they may each be made up of a plurality of pixels merged together into a single mark.
The pattern and the content may be printed substantially simultaneously in a one-pass printing process, i.e. on a single pass of a carrier through the printer. This allows the product, which may be a document, label, packaging article, or any other printed product, to be printed on demand on ordinary plain paper, card or other carrier material. Alternatively the content and the pattern may be printed onto the product separately, for example the content may be printed onto the product which has already been printed with the position identifying pattern.
The present invention is particularly suitable to monochrome printing. However, it can also be used with colour printers, and may indeed be advantageous under some circumstances. For example, colour printers can often be set to print in grey scale, which causes them to mix the different coloured toners, such as cyan, magenta and yellow, to produce different shades of grey. When operating in this mode colour printers can advantageously be operated according to the invention. Also where a colour printer has run out of one or more ink colours it may become necessary to print the content and the position identifying pattern using the same colour, for example to print some of the content in black ink as well as the pattern. Again, in these circumstances, the present invention can usefully be used.
The density of the content elements, which may for example be measured as the total area of content elements per unit area of the image, may be greater than the density of the pattern elements, which may be measured as the total area of the pattern elements per unit area of the image. As the density of the content elements increases the visibility, to a human reader, of the content over the pattern increases, but the ease with which the pattern can be machine read by a reading device, such as a digital pen, decreases. For example, where the content is to be applied as a grey scale, the density may be measured as the grey scale of the content. This is particularly applicable to monochrome printing methods. Where colour printing or marking methods are used for the content, the density may be defined as the average reflectivity of the defined content within a particular wavelength. For example if the pattern is to be produced in some regions using a marking material having a reflectivity in a particular wavelength, then the density can be defined as the average reflectivity of the content within that range of wavelengths. Other measures of density may also be used. For example, where the content is to be applied as a grey scale, the density may be measured as the grey scale of the content. This is particularly applicable to monochrome printing methods. Where colour printing or marking methods are used for the content, the density may be defined as the average reflectivity of the defined content within a particular wavelength. For example if the pattern is to be produced in some regions using a marking material having a reflectivity in a particular wavelength, then the density can be defined as the average reflectivity of the content within that range of wavelengths.
The minimum possible contrast between the individual pattern marks and the content, which allows the reading device to detect the pattern, depends on various factors relating to the reading device, including the resolution of its imaging device and the processing methods it uses to analyse the pattern. According to a second aspect of the invention there is provided a corresponding system for generating an image.
According to a third aspect of the invention there is provided a product having a position identifying pattern and a content feature applied to it, wherein the pattern comprises a plurality of discrete pattern marks each being of at least a predetermined size, the content feature comprises content marks, the content and the pattern are superimposed on each other within at least an area of the product, said area having two dimensions, and within said area the content marks are smaller than the pattern marks in at least one of the dimensions.
According to a fourth aspect of the invention there is provided a method of analysing a position identifying pattern on a product, the product having thereon the position identifying pattern comprising a plurality of pattern elements and a content feature comprising a plurality of content elements, the content elements being smaller than the pattern elements, the method comprising the steps of forming an image of an area of the pattern and the content, and processing the image to extract the pattern from the content on the basis of the relative sizes of the pattern elements and the content elements.
A corresponding system for analysing a position on a product is also provided.
According to a further aspect of the invention there is provided a data carrier carrying data arranged to control a computer system to operate as a system according to the invention, or to carry out the methods of the invention.
The data carrier can comprise, for example, a floppy disk, a CDROM, a DVD ROM/RAM (including +RW, −RW), a hard drive, a non-volatile memory, any form of magneto optical disk, a wire, a transmitted signal (which may comprise an internet download, an ftp transfer, or the like), or any other form of computer readable medium.
Preferred embodiments of the present invention will now be described by way of example only with reference to the accompanying drawings.
The pen 8 comprises a writing nib 10, and a camera 12 made up of an infra red (IR) LED 14 and an IR sensor 16. The camera 12 is arranged to image a circular area of diameter 3.3 mm adjacent to the tip 11 of the pen nib 10. A processor 18 processes images from the camera 12 taken at a specified sample rate. A pressure sensor 20 detects when the nib 10 is in contact with the document 2 and triggers operation of the camera 12. Whenever the pen is being used on an area of the document 2 having the pattern 6 on it, the processor 18 can determine from the pattern 6 the position of the nib 10 of the pen whenever it is in contact with the document 2. From this it can determine the position and shape of any marks made on the patterned areas of the document 2. This information is stored in a memory 22 in the pen as it is being used. When the user has finished marking the document 2, this is recorded in a document completion process, for example by making a mark with the pen 8 in a send box 9. The pen is arranged to recognise the pattern in the send box 9 and send the pen stoke data to a pen stroke interpretation system in a suitable manner, for example via a radio transceiver 24 which provides a Bluetooth radio link with an internet connected PC. Suitable pens are available from Logitech under the trade mark Logitech lo.
Referring back to
The density, or grey scale, of the content dots can vary up to a certain limit, above which the pen 8 is unable to reliably read the pattern 7. Using the normal grey scale where 0 represents black and 255 represents white, a grey scale of from 255 down to about 200, which represents about 30% coverage of black ink on a white carrier, can be used with the pen 8.
The printer 202 can be any printer which has sufficient resolution to print the pattern dots 30 and the content dots 50. In this case it is a 1200 dots per inch (dpi) monochrome laser jet printer. It will be appreciated that the dimensions of the content dots 50 correspond to the dimensions of single pixel of ink from a 1200 dpi printer, and that the spacing between the content dots 50 is twice the spacing of the printer pixels. This enables the printer to print the content dots 50 as single ink dots and the pattern dots 30 as groups of ink dots, for example about 12 dots. The printer dots are not exactly circular but each comprise an irregular mark of ink on the document 2. However the exact shape of the content dots 50 is not important as the human eye cannot see their shape, and the pattern dots 30, because they are made up of a group of printer dots, are close enough to a regular shape to be read by the pen 8. Because they can be distinguished by the pen 8 by virtue of their size, the pattern dots 30 and content dots 50 can be printed using the same type of ink from the monochrome printer. Where a colour printer is used, the ink which is used for the pattern, which would typically be a black ink, can also be used for part of the content where appropriate.
In order to produce the printed document 2 the processor 210 retrieves an electronic document 218, which may be in the form of a PDF file, from the memory 216 and sends it to the printer driver together with instructions as to whether it is to be printed with pattern or not. The electronic document 218 contains a definition of the content 7, and the areas of the document 2 which can have the pattern 6 printed on it. The printer driver then determines from the instructions received whether the document is to be printed with pattern or not. If the document is to be printed without pattern on it, the content is sent for printing. If the document is to be printed with pattern on, the printer driver converts checks the nature of the content to determine whether it is already made up of graphical elements of a suitable format to enable the pattern to be read when the pattern and content are superimposed. If the content is already made up of suitable graphical content elements, then the printing process can proceed. If the content is not suitable made up, for example if it includes areas of solid black, then it is converted so that it is made up entirely of content elements 50 as described above.
When it is determined that the content is all in a suitable format, the printer driver 214 requests the required amount of pattern from the pattern allocation module 212 which allocates by means of coordinate references an area of the pattern space to the document, generates the pattern 6 for that area using a pattern generation algorithm, and communicates the details of the pattern including the positions of all the required dots, back to the printer driver 214. The printer driver 214 then combines the content 7 and the pattern 6 into a single electronic file. This file therefore forms a combined electronic definition of both the pattern and the content. The printer driver then converts the content 7 and the pattern 6 to a format, such as a postscript file, suitable for the printer 202, and sends it to the printer which prints the content 7 and the pattern 6 simultaneously in a one-pass process, i.e. on a single pass of the paper, on which the document is printed, through the printer.
In practice the various components of the system can be spread out over a local network or the internet. For example the pattern allocation module 212 can be provided on a separate internet connected server so that it can be accessed by a number of users.
While the use of a 1200 dpi printer is described above, a similar result can also be achieved with lower resolution printers, such as 600 dpi printers. For a 600 dpi printer, the approximate diameter if each ink dot is 42 μm. This is therefore still well below the minimum diameter for a dot that will be recognized by the pen 8 as a pattern dot. Therefore if the content is printed as single, spaced apart ink dots or pixels from a 600 dpi printer, and the pattern dots are printed as groups of ink dots, then the content and pattern can be printed simultaneously on a 600 dpi printer. Again the grey scale of the content dots needs to be kept at such a level that it will not interfere with the pens ability to identify the pattern dots. A maximum of about 30% grey has been found to work with the Logitec lo™ pen.
If other methods of printing, such as offset printing are used, the resolution of the printed pattern and content can be much higher than with inkjet or laser jet printers. This gives greater freedom in the manner in which the content can be produced.
With the format of content and position identifying pattern described above, it is possible to use various image processing techniques within the pen processor 18 to help distinguish the content from the pattern, for a given resolution of the camera 12 in the pen 8. Because the content dots 50 are smaller than, and closer together than, the pattern dots 30, spatial filtering can be used to select, from all the marks on the document, those which make up the pattern dots 30. Spatial filtering is typically carried out using Fourier transforms, for example as described in WO 01/75783. Referring to
When this Fourier transform method is used, the ability of the processor 18 in the pen 8 to distinguish the pattern 6 from the content 7 is increased, so the content 7 can be made darker than that shown in FIG. 3. For example the content shown in
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5337361||Jun 1, 1992||Aug 9, 1994||Symbol Technologies, Inc.||Record with encoded data|
|US5841978||Jul 27, 1995||Nov 24, 1998||Digimarc Corporation||Network linking method using steganographically embedded data objects|
|US5852434 *||Dec 18, 1995||Dec 22, 1998||Sekendur; Oral F.||Absolute optical position determination|
|US6098882||Mar 1, 1996||Aug 8, 2000||Cobblestone Software, Inc.||Variable formatting of digital data into a pattern|
|US6164847 *||Sep 9, 1999||Dec 26, 2000||Agfa Corporation||Imaging parameter detection|
|US6502756 *||May 26, 2000||Jan 7, 2003||Anoto Ab||Recording of information|
|US6586688 *||Mar 21, 2001||Jul 1, 2003||Anoto Ab||Information-related devices and methods|
|US6674427 *||Oct 2, 2000||Jan 6, 2004||Anoto Ab||Position determination II—calculation|
|US6689966 *||Mar 21, 2001||Feb 10, 2004||Anoto Ab||System and method for determining positional information|
|US20020150276||Dec 29, 2001||Oct 17, 2002||Chang Kenneth H.P.||Encoding and decoding a message within an image|
|US20020186884||Jun 7, 2001||Dec 12, 2002||Doron Shaked||Fiducial mark patterns for graphical bar codes|
|US20030012455||Jun 26, 2002||Jan 16, 2003||Andreas Olsson||Method and device for data decoding|
|WO2000073983A1||May 26, 2000||Dec 7, 2000||Anoto Ab||Position determination|
|WO2001026033A1||Oct 2, 2000||Apr 12, 2001||Anoto Ab||Position determination - calculation|
|WO2001071643A1||Mar 21, 2001||Sep 27, 2001||Anoto Ab||Position information|
|WO2001075773A1||Mar 21, 2001||Oct 11, 2001||Anoto Ab||Information-related devices and methods|
|WO2001075783A1||Mar 21, 2001||Oct 11, 2001||Anoto Ab||Identification of a virtual raster pattern|
|WO2002082366A1||Apr 4, 2002||Oct 17, 2002||Anoto Ab||Method for processing information|
|1||"Anoto Pattern & Digital Paper" Internet: <<http://www.anotofunctionality.com/navigate.asp?PageID=73>> 1 page total (Jan. 19, 2004).|
|2||"Construction" Internet: <<http://web.archive.org/web/20030623220554/www/anoto.com/?url=/technology/patter . . . 1 page total (Jan. 19, 2004).|
|3||"How does it work?" Internet: <<http://www.anotofunctionality.com/navigate.asp?PageID=24>> 1 page total (Jan. 19, 2004).|
|4||"The Technologies Behind Anoto Functionality" Internet: <<http://web.archive.org/web/20030622071228/www.anoto.com/?url;=/technology/1 page total (Jan. 19, 2004).|
|5||Anoto Home Page Internet:: <<http:://www.anoto.com>> pp. 1-2 (Jan. 19, 2004).|
|6||U.S. Appl. No. 10/660,323, filed Sep. 10, 2003, Nelson.|
|7||U.S. Appl. No. 10/660,324, filed Sep. 10, 2003, Nelson et al.|
|8||U.S. Appl. No. 10/661,001, filed Sep. 10, 2003, Mackenzie et al.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7134606 *||Mar 8, 2005||Nov 14, 2006||Kt International, Inc.||Identifier for use with digital paper|
|US7245483 *||Jan 27, 2004||Jul 17, 2007||Satori Labs, Inc.||Integrated personal information management system|
|US7322524 *||Jun 23, 2005||Jan 29, 2008||Silverbrook Research Pty Ltd||Graphic design software using an interface surface|
|US7503493||Nov 25, 2005||Mar 17, 2009||Silverbrook Research Pty Ltd||Method and system for digitizing freehand graphics with user-selected properties|
|US7568622||Jun 1, 2007||Aug 4, 2009||Silverbrook Research Pty Ltd||Input unit arrangement|
|US7773377 *||Aug 10, 2010||Satori Labs, Inc.||Integrated personal information management system|
|US7815305 *||Jan 12, 2005||Oct 19, 2010||Koninklijke Philips Electronics N.V.||Electronic paint brush with scanner and dispensers|
|US7832640||Feb 1, 2009||Nov 16, 2010||Silverbrook Research Pty Ltd||System for digitizing freehand graphics on a printed surface|
|US7934655||Jul 12, 2009||May 3, 2011||Silverbrook Research Pty Ltd||Input arrangement for controlling electronic device|
|US7934656||Oct 14, 2010||May 3, 2011||Silverbrook Research Pty Ltd||Digitizing system having printed base, imaging pen and relay device|
|US8091787||Jan 10, 2012||Silverbrook Research Pty Ltd||Processing system for digitizing freehand graphics|
|US8130391 *||Aug 27, 2004||Mar 6, 2012||Hewlett-Packard Development Company L.P.||Printing of documents with position identification pattern|
|US8692212||Dec 13, 2012||Apr 8, 2014||3M Innovative Properties Company||Optical digitizer system with position-unique photoluminescent indicia|
|US9068845||Dec 13, 2012||Jun 30, 2015||3M Innovative Properties Company||Optical digitizer system with position-unique photoluminescent indicia|
|US9075452||Feb 12, 2014||Jul 7, 2015||3M Innovative Properties Company||Optical digitizer system with position-unique photoluminescent indicia|
|US9235743 *||Apr 25, 2014||Jan 12, 2016||Fujitsu Limited||Image generating method, information processing method, and information processing device|
|US20040101278 *||Nov 17, 2003||May 27, 2004||Canon Kabushiki Kaisha||Recording apparatus capable of recording information representing positions on a recording medium and recording method|
|US20050013104 *||Jan 27, 2004||Jan 20, 2005||Satori Labs, Inc.||Integrated Personal Information Management System|
|US20050211783 *||Mar 8, 2005||Sep 29, 2005||Henwell Chou||Identifier for use with digital paper|
|US20050243373 *||Jun 23, 2005||Nov 3, 2005||Sliverbrook Research Pty Ltd||Graphic design software using an interface surface|
|US20070086032 *||Aug 27, 2004||Apr 19, 2007||Hewlett-Packard Development Company L.P.||Printing of documents with position identification pattern|
|US20070229511 *||Jun 1, 2007||Oct 4, 2007||Silverbrook Research Pty Ltd||Input Unit Arrangement|
|US20080012839 *||Jun 29, 2007||Jan 17, 2008||Satori Labs, Inc.||Integrated Personal Information Management System|
|US20080114777 *||Aug 27, 2004||May 15, 2008||Hewlett-Packard Development Company, L.P.||Data Structure for an Electronic Document and Related Methods|
|US20080239333 *||Mar 27, 2008||Oct 2, 2008||Oki Data Corporation||Printing system|
|US20090135163 *||Feb 1, 2009||May 28, 2009||Silverbrook Research Pty Ltd||System For Digitizing Freehand Graphics On A Printed Surface|
|US20090141112 *||Jan 12, 2005||Jun 4, 2009||Koninklijke Philips Electronics N V||Electronic paint brush with scanner and dispensers|
|US20090267915 *||Jul 12, 2009||Oct 29, 2009||Silverbrook Research Pty Ltd||Input arrangement for controlling electronic device|
|US20090309854 *||Jun 13, 2008||Dec 17, 2009||Polyvision Corporation||Input devices with multiple operating modes|
|US20110024492 *||Feb 3, 2011||Silverbrook Research Pty Ltd||Digitizing system having printed base, imaging pen and relay device|
|US20110174882 *||Jul 21, 2011||Silverbrook Research Pty Ltd||Processing system for digitizing freehand graphics|
|US20140035880 *||Oct 8, 2013||Feb 6, 2014||Panasonic Corporation||Display control system, pointer, and display panel|
|US20140231522 *||Apr 25, 2014||Aug 21, 2014||Fujitsu Limited||Image generating method, information processing method, and information processing device|
|U.S. Classification||400/76, 400/74, 178/18.01, 235/494|
|International Classification||G06F3/03, G06F3/033|
|Cooperative Classification||G06F3/03545, G06F3/0321|
|European Classification||G06F3/0354N, G06F3/03H3|
|Apr 12, 2005||AS||Assignment|
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HEWLETT-PACKARD COMPANY;REEL/FRAME:016050/0887
Effective date: 20050412
|Apr 25, 2005||AS||Assignment|
Owner name: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P., TEXAS
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HP CENTRE DE COMPETENCES FRANCE S.A.S.;BROUHON, PATRICK;REEL/FRAME:016150/0245
Effective date: 20040621
Owner name: HEWLETT-PACKARD COMPANY, CALIFORNIA
Free format text: COPY OF EMPLOYEE AGREEMENT OF IRA GOLDSTEIN WITH HEWLETT-PACKARD COMPANY;ASSIGNOR:GOLDSTEIN, IRA;REEL/FRAME:016150/0248
Effective date: 19810803
|May 8, 2009||FPAY||Fee payment|
Year of fee payment: 4
|Oct 20, 2009||CC||Certificate of correction|
|Mar 8, 2013||FPAY||Fee payment|
Year of fee payment: 8