US 20070016578 A1
A system for the support and management of search for documents is presents. The system includes knowledge-database, query interface and communication to a database of documents to be searched. Information generated during a search session is collected by the system and is added to the knowledge-database. The information is ranked automatically according to the usage of that information by the user. During successive search session, or during search made by other users, the system uses the knowledge-database to support the users with keywords, queries and reference to documents.
1. A method of linking documents to queries, comprising the steps of:
submitting a query for a search process;
displaying a list of references to documents in response to said submission of said query;
tracing at least one activity of the user in reference to said displayed list of references;
creating a link between said query and at least one reference to a document that is interacted by the user; and
storing the link between said query and said at least one reference.
2. The method of
3. A method according to
a. a link to a documents; and
b. a document;
said ranking comprising the steps of:
displaying the submitted query;
displaying at least one linked references to documents; and
increasing the rank of said at least one item if any of the following is executed by the user:
a. said document is opened;
b. said document is printed;
c. said document is stored in a storage device;
d. said document is sent, through computerized means, to another user; and
e. the user has clicked a button indicating his wish to increase the ranking of said at least one item.
4. The method according to
5. A method according to
6. A method according to
a. opening and closing the document without additional action on the document; and
b. clicking a button indicating the user's wish to decrease that ranking of said at least one item.
7. The method according to
8. A method of retrieving documents in a document search process, comprising the steps of:
submitting a first query for a search process;
using said first query to search a query storage for second queries containing at least one keyword of said first query;
retrieving a list of links from said second queries to linked references; and
displaying a list of said references.
9. A method according to
sorting said references in groups according to the ranking of said second queries;
sorting the references in each group according to the ranking of the said linked references; and
displaying a list of said references in the sorted order.
10. A method of sorting references to documents for displaying a list of references in a search process, comprising the steps of:
selecting, by the user, to display one of:
a. references to documents displayed in the past to the user; and
b. references to documents that were never displayed to the user;
for each reference suggested by the search process, checking whether the document of said reference has been displayed before to the user; and
determining the handling of said reference, in conjunction to said displayed reference list according to said user selection.
11. The method according to
displaying a first group of references to documents that have been displayed in the past to the user, said documents' rankings being above a pre-determined value; displaying a second group of references to documents that have not been displayed to the user; and
displaying a third group of references to documents that have been displayed in the past to the user, said documents' rankings being below a pre-determined value.
The invention is related to the field of digital information management and, in particular, to search and retrieval of the same.
The growth of the Internet introduced broad access to information not available before. Access to digital information sources, once an asset of a small group of professional, holding information technology skills, has become a daily tool for millions of the Internet users.
Lower skill levels for search of such information is now an inseparable part of searches made for information by most of the users. Skill and knowledge aspects became limiting factors for successful search and retrieve of such information. Such limitations present themselves in the composition of Boolean queries, knowledge of relevant information sources such as the AltaVista index at www.altavista.digital.com (maximum coverage of Internet documents by a single index is less then 40%). Many of the documents are accessed only by a direct reference from a person with a specific knowledge. Many of the information provided by querying an index such as Yahoo are not relevant. The search process is long, queries are modified many times and many information pieces are missed.
Natural language query is one suggested replacement for Boolean query method. However, a research from Search Insider (www.searchinsider.com) indicates clearly that users prefer the Boolean query methods and that search engines such as AltaVista and Yahoo (www.yahoo.com) provide require much improvement.
The present status of information search and retrieval in the Internet is characterized by long and tedious search process, poor relevancy of retrieved documents and low level of success in retrieval of relevant documents.
It is the purpose of the present invention to provide a method and tools for efficient search and retrieval of documents. The description in reference to the Internet is maid in a way of example only. It would be appreciated by those skilled in the art, that the proposed method is applicable to any digital information source.
In a preferred embodiment of the present invention, the system comprises a Server, Clients having a Human Interface (H/I) and communication with the server, a Database, a connection to the Internet and Software of that system. A User of the Client is using the H/I to compose a query view various information pieces including retrieved documents or titles or summary of those documents. The Software traces the user's search activity, collecting data such as queries and relevant or irrelevant documents and information sources. The data is evaluated and stored in the Database.
The data is also compared to other data, collected from all users, available from the Database. A data that is relevant to assist the current search process is presented to the user. This data include highly ranked queries that are similar to the query composed by the current user and highly ranked documents retrieved in the past by those queries.
As the Database is the accumulation of many such search processes, made by many users, in the same subject, the current user can benefit from the many hours invested by others to find information in this subject.
This method is particularly effective when used within an organization, whereas the organization members search for information that is relevant to the activity of that organization. Thus the Database generated overtime is highly relevant to all members of such organization.
The invention will be better understood in reference to the following Figures:
Reference is made now to
1. Server 102 which is any suitable computer such as IBM Netfinity 3000, available from IBM Corporation, North Carolina, U.S.A. with a suitable server software (Server S/W 106) such as Microsoft Windows NT, available from Microsoft Corporation, Washington, U.S.A.
2. At least one Client Station 104 which is any personal computer such as IBM PC 300XL with suitable software (Client S/W 112) such as Microsoft Windows NT (the drawing illustrates multiple Client Stations 104).
3. Server 102 includes also System Database 108.
4. Each of Client Stations 104 is connected to the Server by any means of communication 110 (such as LAN) and any standard communication protocol (such as TCP/IP).
5. Each of the Client Stations contains also Human Interface 114, such as described in details in reference to
6. System 100 is also connected to at least one source of digital information 116 by communication means 118. In the preferred embodiment presented here, such a source of digital information is the Internet. This connection may be done using any method, some of the methods are described in “Using the Internet”, 2nd edition by Mary Ann Pike, ©1995.
Initial Search Process:
Initial search is characterized by no data in System Database 108. At this stage System 100 can not assist the user to find the information he is looking for. In this stage System 100 traces the user's search session, collects data from the search session, processes the data and stores the results in the database.
In the present embodiment of the invention, the user composes a Boolean query using Human Interface 114. The user then submits the query to a search engine such as AltaVista. A set of titles and summaries in presented to the user by the search engine.
The user may now select some of the titles for a detailed review and ignore the others.
When the detailed document is presented to the user it is desired that the evaluation of the document, by the user, will be provided to the System. For that purpose a window is displayed with the document. The window contains a dual button interface indicating Y and N. To change from the document display back to the search session the user must click either the Y or the N. Y indicates that the document is relevant to the user and N indicates irrelevant document. System 100 also traces activities such as printing a document or saving a document on disk. These actions are considered as parts of positive evaluations of a document.
The user, repetitively composes improved queries, submits them to the search engine and evaluates results (direct or indirect evaluation). During the search session the user may submit queries to different sources such as AltaVista, Yahoo and Lycos (www.lycos.com). Also meta-search engines such as MetaCrawler (www.metacrawler.com) may be used.
When the user finishes the search session, the system has collected raw data ready for process. This data includes details such as the following:
1. Queries submitted by the user.
2. URLs of documents (URL: Uniform Resource Locator)
3. One query to many URLs relations for the URLs retrieved by each one of the submitted queries.
4. A set of evaluation parameters for each URL:
5. Date and time session ended.
6. Register indications of who, within the users group, reviewed the document.
7. Date URL introduced to System Database.
A process of the data is begun at this stage to produce a set of valuable information in the System Database, the information contains details such as:
1. Ranking each of the URLs as per the following example:
This ranking method enables increase of URL ranking either by direct or indirect evaluation by the user. Also, Rank reduction effectiveness is limited to avoid unintentional negative ranking of a URL.
2. A ranking method for a query is provided hereinbelow as an example: Generate ranking parameters for each submitted query:
‘Term’ is used here in the broad sense. For example, a ‘term’ may be a single word, expression containing more then one word and a sequence of characters with indication to refer to all words and expressions containing that sequence of characters.
A query may then be ranked using a formula such as:
This formula may use a relatively high value for K1 to make valuable URLs more effective in ranking the query as a useful query. K2 may be a relatively low value to limit the effect of irrelevant URLs retrieved by this query. This satisfies the need to reduce the rank of a query if many irrelevant URLs are retrieved by the query but, at the same time, make sure that the query maintains a good rank if it also retrieves valuable URLs.
T1 is an indication for the sophistication of the query that adds to the value of such query to the user.
It would be appreciated that ranking formula (1) is an example and many other formulas may produce valuable solution.
3. Generating a set of related terms: each query is analyzed for terms related by Boolean operators such as OR, AND and NOT. This generates a many to many relations among terms of the query.
Consider the following query example:
(car or automobile or vehicle not bus) and (road or sand or mud).
The following relation table may be generated:
This vocabulary, with the Boolean relations, is useful in identifying terms that are relevant to other terms, in the sense of composing a Boolean query. For example, when the word ‘vehicle’ is introduced in a query, the System may use the System Database to present the user with other terms that may be useful, under the relevant category:
The user may then use these related terms to enhance his query.
4. Ranking each of the terms. The terms may be ranked according to the number of times they appear, in relation to a given term, in all queries. The ranking may be separate for each type of Boolean relation, such as OR, AND and NOT.
This ranking may also be modified, by weighting each appearance by the rank of the query, that generates said relation.
The data collected during the search session is then stored in the System Database together with the set of parameters and calculated ranks. This also includes submitted queries, URLs, summaries, terms and their relations and ranking.
The amount of such data accumulates more rapidly when the System is multi user. This is the case in organizations, where many members of the organization search for information regarding similar subjects.
Reference is made now to
In order to enable a comprehensive explanation, it is assumed that an amount of data has already been collected in System Database 108 by the process described hereinabove and this data is available for assistance of searches made by individuals.
Query Composition Section:
In a preferred embodiment of the invention, Query Composition section 200 contains Query window 202 where the current query is composed and Related Queries window 204 for display of relevant queries stored in System Database 108.
Basic query composition is done in Query window 202, by typing words, expressions, Boolean operators and conventional singes. Following are some examples:
Related queries are displayed in Related Queries window 204. In a simple implementation of the present invention, terms from Query window 202 are used to search System Database for stored queries that contains such terms. The queries are displayed in Related Query window according to their rank, highest rank first.
In a more sophisticated implementation of the present invention, the queries containing terms of Query window 202 are specially ranked for display in Related Queries window 204. This ranking may first rank the queries in accordance to the number of terms from Query window 202 that are contained in these queries and then, a sub-level sorting is performed, where all queries of the same ranking are sub-sorted by their own rank (as stored in System Database 108). Other ranking schemes are possible.
In default, the top query of Related Query window 204 is highlighted. Any other query of window 204 can be highlighted by clicking on it. More then one query may be highlighted simultaneously. Double clicking a query of Query window 204 will copy the query to Query window 202, instead of what was previously in that window. Related Query window 204 will refresh then, according to the new query in Query window 202.
More queries may be observed in Related Queries window 204 by using Roll Slider 236.
Related Terms section 206 within Query Composition section 200 supports the process of query composition by listing terms that are related to the terms of the query in Query window 202. Related Terms section 206 contains four windows:
2. ‘Or’ window 210 contains terms that we are related to the terms of Key Terms window 208. The relation is established by the Boolean operator ‘OR’ in queries composed and stored in System Database 108 in previous search sessions. For example ‘Term1’ in window 210 appears in at least one query stored in System Database 108,
3. ‘And’ window 212 contains terms that we are related to the terms of Key Terms window 208. The relation is established by the Boolean operator ‘AND’ in queries composed and stored in System Database 108 in previous search sessions. For example ‘Term4’ in window 212 appears in at least one query stored in System Database 108,
4. ‘Not’ window 214 contains terms that we are related to the terms of Key Terms window 208. The relation is established by the Boolean operator ‘NOT’ in queries composed and stored in System Database 108 in previous search sessions. For example ‘Term7’ in window 214 appears in at least one query stored in System Database 108,
Nested Boolean relations are supported. For example, query of the type
will generate the following relations:
Button ‘Go’, 216, is used to indicate that the composition of the query is complete and the system is to retrieve information in accordance to that query.
Document section 218 is used to display titles and summaries of documents in accordance to the highlighted Query in Related Queries section 204.
The titles of the documents are displayed in Title window 220 are titles that are highly relater to the highlighted query of window 204, Query2 in the example of
The titles are available from System Database 108 and are displayed in order according to their ranking. High rank displayed first.
If the number of such titles is lower then No. of Titles 228 (20 in the example of
Titles are also filtered for display by Which Title? window 232. In the example of
A single click on a title of window 220 will effect the display of summaries in Summary window 222 as explained hereinbelow.
A double click on a title of window 220 will display the document of that title. Window 224 is an indicative window that displays the URL of the highlighted title of Window 220.
The summaries of the documents that are displayed in Summary window 222 are related to the list of titles in window 220. The summary of the highlighted title, Title3 of window 220 in the example of
Double clicking a summary will display the document of that summary.
Preferences section 226 is used for input, by the user, of preferences for the search session. A maximum number of titles to be presented is indicated in window 228, adjustable by the user. Date Range windows 230 indicate the earliest and latest dates of that rang. Only documents that were updated in System Database 108 within this range will be presented. The dates are adjustable by the user. Which Title window 232 contains multi-choice options. Three such options may be:
Database windows 234 indicates the database to be used by the system.
Local window 238 is used to select the preferred local database to perform as System Database 108. This may be one of few local databases or a portion of a database. The different options may be indicated by names such as:
This arrangement is described in more details in Israeli patent application number 119183 dated Sep. 2, 1996 (Haim Zvi Melman et al.).
The selected database is used during the user's interaction with the human interface of
Internet window 240 is used to select the preferred Internet database sources. This is useful for submission of queries for search of new documents (or titles) that are not in the local database. Pre-configured selections of sources are available at this window.
Aided Search Session:
The aided search session will be described hereinbelow in reference to
For simplicity, all elements of
All elements of
It will be appreciated that the system can handle multiple simultaneous search sessions, conducted by different users, independently. Each user may use his own preferences configuration in Preferences window 226. The data collected from each user is added to System Database 108 to serve all users. Parts of System Database 108 may be limited for use by only few users, not accessible to others.
In this example, the preferences of window 226 will be as indicated in
The user may view documents, step back to the search process modifying his search activity in a variety of ways such as composing new queries, editing queries and changing preferences of window 226.
Following predetermined conditions (such as every 10 minutes or end of search session or every logout) System 100 is processing the new search information and generates new data for System Database 108. This data is added to the previously stored data for future aided search sessions, as described hereinabove (section of Initial search process).
Simple queries such as of one or two Key Terms are the most intuitive and are frequently the first ones composed in a search session. In another embodiment of the invention, the user may submit such a simple query that in a present art search system will fail to produce useful results and provide many falls results. In the present invention, submission of such a query will actually result in submission of a number of more sophisticated queries, these are Related Queries available from System Database 108. These queries may be those of a rank above a predetermined threshold. The results will be those that are associated with the highly ranked Related Queries. This methods enables valuable results from a simple and intuitive query—not effective by itself.
A method is presented hereinbelow, to associate such simple queries that, in most cases, do not provide the desired results, with more sophisticated queries that are usually not intuitive and are composed only after few interaction with search results and query modifications accordingly.
A search session is defined to be the process of searching information related to a specific subject. A search session includes the composing of queries, submission of queries to a search engine, evaluation of results, modification of queries as a response to such search results, submission of such modified queries—and so on.
Two sessions are different if the subject is different.
In the present embodiment of the invention, semi-automatic session tracing is performed by the system. Queries are identified to belong to the same search session in the following way:
New search session starts by a query (normally after the application is activated). Each new or modified query is compared to all the previous queries. If at least one keyword of this query is used in one of the previous queries of that session—the new query is belongs of the same session.
If the user go through strategy change in his queries there may be no keyword relation anymore to previous queries even if this is the same search session. For example, a user may search for information on 35 mm film dimensions. He may start with a query “film and 35 mm” and change strategy to search for standard organizations with the query “iso”. To overcome this problem in the present embodiment, when this occurs, the system responds to the user with a question: “Have you started a new search session? Y/N”. If the user answers “N” the queries “film and 35 mm” and “iso” are associated with the same search session. Next time that the query “film and 35 mm” will be composed by a user, a reference will also be made to the query “iso” as a relevant query. Useful titles retrieved by the query “iso” will be available then also for the query “film and 35 mm”. If the user replies “Y”, the previous set of queries are associated with one search session while the new query is the first one of the new search session.
In yet another embodiment of the invention, the data that is collected and processed trough the search sessions is used to generate and support direct information exchange and communication among the members of an organization.
As described hereinabove, during the search session Related Queries and Titles from System Database 108 are associated with the search activity of the user. Since such queries and titles are related also to other users who used them in the past, the names of such users can be presented to the current user for user to user communication.
In the present example, after the user has worked through at least a part of his search session, such a relation to Related Queries and Titles of System Database 108 has been established. The user interface of
The order of names in the window may by according to the order of Related Queries of window 236 of
The names are adjacent to checkboxes. The user may now, for example, use the checkboxes to select a subgroup of the names and then select an Email option. An email form addressed to the checked names will be opened. The user may then write an email to this group of user, asking for more information about their activity in the subject of his search session.
Of course, the user may also select to use the phone numbers to call these people.
An example that highlights the usefulness of this method is provided hereinbelow: A user may look for an international standard in a specific matter. By looking at the ISO Internet site (www.iso.ch) He may find a reference to the standard he needs but the standard itself is not available there. The standard has to be ordered separately, be paid for and delivered by mail.
In this example the user may place a relevant query in query window 202 in order to get names of people, in the organization, who are involved in that subject. Then he can email them an inquiry to find whether they have in possession the requested document. By doing so he may shorten the time and cost to get the required document.
It would be appreciated by those skilled in the art that efficiency of System 100 depends on the computers in use, communication networks and other device parameters.
The flow of process, as described hereinabove may be modified to suit less efficient devices by avoiding updating the windows of
It is also appreciated that non-Boolean query systems, such as Natural Language Queries, may be used in the present invention.
The hereinabove embodiments are described in a way of example and do does not specify a limited the scope of the invention.
The scope of the invention is defined solely by the claims provided hereinbelow: