|Publication number||US6886019 B1|
|Application number||US 09/571,673|
|Publication date||Apr 26, 2005|
|Filing date||May 15, 2000|
|Priority date||May 15, 2000|
|Publication number||09571673, 571673, US 6886019 B1, US 6886019B1, US-B1-6886019, US6886019 B1, US6886019B1|
|Inventors||David Maxwell Cannon, Colin Scott Dawson, Howard Newton Martin|
|Original Assignee||International Business Machines Corporation|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (20), Referenced by (20), Classifications (20), Legal Events (6)|
|External Links: USPTO, USPTO Assignment, Espacenet|
1. Field of the Invention
The present invention relates generally to data processing systems, and more particularly to storage management servers for optimizing selection and accessing of stored files to avoid mount and position thrashing.
2. Description of Related Art
Data processing systems typically require a large amount of data storage. Customer data, or data generated by users within the data processing system, occupy a great portion of this data storage. Effective data processing systems also provide backup copies of this user data to prevent a loss of such data. Many businesses view any loss of data in their data processing systems as catastrophic, severely impacting the success of the business.
A storage management server provides an effective means for protecting customer data. Generally, a client-server configuration includes several clients connected to a single server. The end users create client files and transfer these files to the server. The server receives the client files and stores them on attached storage devices. When used as a storage management system, the server manages the backup, archival, and migration of these client files. By storing the client file on an attached storage device, the server creates a first, or primary, copy of the client file. The server may, in turn, create additional backup copies of the client file for inclusion in the overall storage hierarchy to improve the data availability and data recovery functions of the storage management system. Clients may vary from small personal computer systems to large data processing systems having a host processor connected to several data storage devices. The server can also range from a small personal computer to a large host processor.
An advanced storage management server, such as Tivoli Storage Manager (formerly known as ADSM), maintains reference information about the client files copied within the attached storage volumes. The server uses a database to keep inventory information about the original client files and storage volume location information about the copies of the client files stored within the server. The inventory information typically includes a client system identifier, a client system directory, a client file name, and other attributes of the file. The location information typically consists of a storage volume identifier and a position within the storage volume among other storage attributes. In addition, the server database allows the server to assign a unique identifier to each client file stored within the attached storage volumes. Thus, the server can track individual files throughout the server storage component.
Accordingly, the server database introduces several advantages to the storage management server. The server can track multiple copies of an individual client file written to different storage volumes. By tracking secondary copies of the client file, the server improves the data availability to the client systems. For example, if a primary copy of a particular client file is inaccessible because it is stored on a destroyed volume or damaged media, the server can access an additional copy residing on a different storage volume and transfer the additional copy to the requesting client system. Further, the server can subsequently recover the unavailable primary copy of the client file from the secondary copy. The server needs both inventory and storage volume location information provided by the server database to accomplish the above-described data recovery.
A data processing system using a storage management server, including a file storage manager, stores files that have been backed up or archived from various client nodes. The server stores client data files in a storage hierarchy consisting of various media types (e.g., magnetic disk, tape, optical disk) and uses a database for tracking the attributes and storage location of each stored client file.
Another function of a storage management server is to select files that satisfy certain criteria, and transfer the files to another location. There are many situations in which the transfer of data to another location is necessary or desirable. For example, it may be desired to create a backup set that represents the latest set of files stored on the server for a particular client node. The backup set could be used for restoring files directly to a client node, without requiring use of a network, or for transporting these files to another server. Those skilled in the art will recognize that the creation of a backup set is only one example and that other applications are well-suited to the copy or transfer of data from one location to another. In general, the specification will refer to copied files as belonging to a copy set.
In this copying operation, data on the source server may be stored on various types of media or volumes. For example, storage media can be removable or non-removable, and can be accessed either sequentially or randomly. Typically, a storage management server can process files from different volumes types. For example, it can process data from random-access, non-removable volumes which do not have to be mounted each time they are accessed and are randomly searched; sequential volumes, such as tapes, which are mounted at the beginning of the volume and are sequentially processed; and random-access, removable volumes, such as optical disks, which are mounted for each search but randomly processed once mounted.
The description will continue in an illustrative sense with respect to storage volumes, which comprise random-access media and sequential-access media. Random-access media is considered to include media that is both non-removable and random-access. Sequential-access media is considered to include all removable media, whether it is accessed randomly or sequentially.
Information on random-access media, such as magnetic disk, can usually be transferred relatively efficiently. However, transferring data from sequential-access media can impose delays while the required volume is mounted. Moreover, additional delays may be required to position the media to the correct location on the storage volume.
Accordingly, one of the major challenges in generating a copy set or performing any copying operation is to discover how to efficiently copy numerous files from sequential-access media, such as magnetic tapes. The efficient copying should be done with minimal mounting and positioning of input volumes. Therefore, optimized selection and accessing of stored files should avoid mount and position thrashing, which occurs with excessive moving back and forth between the mounted volumes or positions within a volume.
In addition to problems encountered by certain types of media as just described, a further challenge arises from an efficiency problem inherent to the functionality of the copying operation. The problem stems from the utilization of two completely different views of the data, namely the inventory view and the storage view, in the copying operation.
Files are normally selected from inclusion in the copy set based on inventory view attributes of the data, important to the end user. Such attributes include the client node, filespace and file name information, and recency of the copy. As used in this specification, the term “filespace” refers to a logical space in the client's storage that can contain a group of files. For example, a filespace could be a logical partition or a directory and its subdirectories.
On the other hand, efficiency requires that files be transferred in some optimal order that depends on the location of these files within the server's storage hierarchy. This information is part of storage view attributes of the data and relates directly to the various types of storage media previously described.
Conventional solutions to the efficiency problem in utilizing both views typically include creating a list of files based on filter criteria which are evaluated using the inventory view. Files in the list are then sorted by their storage location and are transferred in sorted order, which represents the storage view. However, this approach requires a great deal of initial overhead to create and sort the list of files, which delays the transferring of files.
Therefore, there is a need in the art to provide a means whereby a copy set can be generated in optimal manner by considering both the inventory and the storage view of files, and without creation of a sorted list of files.
To overcome the limitations in the prior art described above, and to overcome other limitations that will become apparent upon reading and understanding the instant specification, the present invention discloses a method, apparatus, and article of manufacture for a computer-implemented technique for generating a copy set in such a way as to minimize mounting and positioning of storage volumes.
In accordance with the present invention, the method receives a copy set generation request specifying selection criteria for files to be included in a copy set, identifies matching files meeting the selection criteria, locates the matching files on their storage volumes, and copies the files to the copy set, ignoring the file order in the request but considering the proximity of the matching files to each other on the storage volumes. The method ensures that each matching file is included, without duplication, in the copy set, and also ensures that the files are copied with minimal delays in mounting and positioning of the storage volumes.
The storage volumes preferably include sequential-access volumes and random-access volumes; and stored files have a primary copy on a sequential-access volume or a random-access volume, and may have a secondary copy on a sequential-access volume. The copying is preferably attempted in the following order:
The apparatus embodiment includes a computer having data storage volumes connected thereto and one or more computer programs, performed by the computer, for executing the above-described method for generating a copy set in such a way as to minimize mounting and positioning of the storage volumes.
The article of manufacture embodiment includes a computer program carrier readable by a computer and embodying one or more instructions executable by the computer to perform the above-described method steps for generating a copy set in such a way as to minimize mounting and positioning of storage volumes.
Referring now to the drawings in which like reference numbers represent corresponding parts throughout:
In the following description of the preferred embodiment, reference is made to the accompanying drawings which form a part hereof, and which is shown by way of illustration of a specific embodiment in which the invention may be practiced. It is to be understood that other embodiments may be utilized as structural changes may be made without departing from the scope of the present invention.
The present invention provides a means whereby a copy set can be generated in optimal manner by considering both the inventory and the storage view of files. Referring more particularly to
Each client system 15 creates original user data files, or client files, which are stored within the corresponding client system 15. The client systems 15 transfer client files to the server system 20. Transferring client files to the server 20 inherently provides a copy mechanism within the server 20 for these original client files. The storage manager 30 directs the client file to an attached storage volume 40. The server 20 stores a first, or primary, copy of the client file on a primary storage volume 40 and may also generate additional copies of the client file on secondary storage volumes 40. The storage manager 30 maintains inventory information about the client file and reference location information pertaining to the copies of the client file within the server database 60.
The server database 60 allows the server 20 to manage individual files within the server storage component 40. The server database 60 introduces advantages to the storage management server 20. The storage manager 30 can track multiple copies of an individual client file written to different storage volumes 40. If the primary copy of a client file is unavailable, the storage manager 30 can access a secondary copy from a different storage volume 40, should a secondary copy be available, using the reference location information in the server database 60. Moreover, the storage manager 30 can recover the primary copy of the client file from a backup copy. In addition, the server database 60 allows the storage manager 30 to coordinate incremental copy operations from a client system 15 to the server 20. The server database 60 denotes which client files have been added to the server storage 40 since a previous incremental copy operation was completed. Without the server database 60, the storage manager 30 must resort to full backup of client data.
The inventory view employs a file inventory list 80, shown in
An inventory view entry 100 is expanded to show a portion of the inventory information. A server inventory entry 100 typically provides inventory information about the client file. In
The storage view utilizes a storage reference list 90, which contains various entries 120, and a storage volume contents list 140 containing entries 150. Each storage reference list entry 120 typically provides reference location information about a particular copy of the client file. A server storage 20 can be organized into sets of storage volumes 40, called storage pools. Each set, or pool, is homogenous with respect to media type, in that a pool contains only media of the same type. A file may be located within the server storage 20 by specifying the storage pool, the storage volume within the storage pool, and the position within the storage volume. The information in the entry 120 may be arranged so as to locate a file associated with a particular bfid as efficiently as possible. Accordingly, the entry 120 may contain information including a storage pool identifier 112, bfid 110, a storage volume identifier 114, and a position 116 within the storage volume, in that order. This is a reasonable exemplary ordering of information in that it directs the search for a file first to a pool 112 according to media type, and then to bfid. It is noted that in addition to providing file identification, the bfid 110 also serves the purpose of mapping the information within the inventory 80 to the reference location information within the reference list 90. Further, if a file spans multiple storage volumes on the server, a separate reference list entry 120 is used for each volume on which the file is stored.
Continuing with the exemplary embodiment of the present invention, once a volume that is known to contain a requested file has been mounted, file searching is directed by the storage volume contents list 140. Information stored in the entries 150 in this list are ordered by location within the volume such that all requested files stored on the volume may be copied in the order in which they appear, regardless of the order in which they are requested. A reasonable ordering of the information in the entries 150, then, would comprise the sequence: volume 114, position 116, and bfid 110. This ordering of information facilitates the efficient copying of all requested files on the mounted volume and ensures that once the method of the present invention begins processing a sequential or removable volume, all files on that volume are transferred in optimal order. The present invention, therefore, minimizes the mounting and positioning of volumes during acquisition of all the requested files in the inventory.
As supported by the layout of FIG. 2 and its associated description, the present invention provides a technique whereby a copy set can be generated in optimal manner by considering both the inventory and the storage view of files. The end result is that files are selected based on filter criteria of the inventory view, but are transferred without excessive mounting or positioning of volumes, according to the storage view. One advantage of the present invention is that the efficiency is achieved even if files must be accessed from secondary locations, due to media defects or other problems. Another advantage of this approach is that file transfer begins almost immediately, without the overhead of first sorting files according to their storage location.
In the present invention the inventory component identifies the matching files that meet the specified filter criteria and ensures that each matching file is included, without duplication, in the copy set. For each matching file, the inventory component invokes the storage component which locates the file in the storage hierarchy and copies the file to the copy set, in such a way as to minimize mounting and positioning of storage volumes.
The efficiency is supported by the key feature of the present invention, according to which the storage component does not necessarily copy files in the requested order. Instead, the storage component may anticipate a future request and transfer a file based on its proximity to other files in the storage hierarchy, even before it has been requested to do so. Alternatively, the storage component may receive a request to copy a file but defer processing if it is not possible to retrieve the file from its primary location. After the method has attempted to transfer all files from their primary locations it begins processing any deferred secondary volumes. This optimization, based on storage volume selection as well as file position within a storage volume, avoids mounting a secondary volume whenever a file cannot be transferred from the currently mounted primary volume. The method also ensures that files will be transferred from their primary location if it is possible to do so.
The inventory and storage components interact in a such a way as to ensure that every matching file is copied to the copy set, and that file transfer is performed in an optimal manner with regard to mounting and positioning of storage volumes.
Accordingly to exemplary embodiments of the present invention, the inventory component flowcharts are presented in
The inventory component uses tables or lists in the server's database for locating files which satisfy the filter criteria given by the selection attributes, as shown in step 202. As the copy set is generated, this component also constructs a temporary table or list which contains an entry for each file that has already been copied to the copy set. The temporary table is used to avoid duplicating the same file within the copy set, but can also facilitate construction of a catalog of files in the copy set.
The inventory component scans the server's database tables, such as the file inventory list 80 in
The inventory component of the exemplary embodiment provides two call-back routines that can be invoked from the storage component. The first call-back routine determines, for any specific file found on a storage volume, whether that file should be added to the copy set. An affirmative response is given if and only if the file satisfies the filter criteria (determined by checking database information against the specific filter criteria) and has not already been added to the copy set (determined by looking up the file in the temporary table).
The second call-back routine provides the ability for the storage component to notify the inventory component that it has successfully copied a file to the copy set. Upon notification, the inventory component adds the file to the temporary table to avoid duplication of files in the copy set.
After the inventory component has identified all matching files and requested that these be added to the copy set, it invokes the storage component one last time to perform any residual processing in step 206 of FIG. 3. This allows the storage component to defer processing of files stored at secondary locations until all other work has been completed.
The storage component provides an entry point that is invoked by the inventory component to request that a specific file be added to the copy set. The storage component does not necessarily satisfy these requests in the order they are received, since that would be inefficient. Instead, the storage component processes the requested file in conjunction with other files that are stored in close proximity.
In view of the problems associated with various types of media discussed in the background herein, there may be considered two classes of media from which data can be transferred according to the present invention. The first class, including media that is removable, sequential, or otherwise inefficient for data access and transfer, is handled in a way to minimize delays and thereby achieve efficiency. The second class, including media that is both non-removable and random-access, is handled in another way because data transfer from such media is inherently efficient. The present invention provides a solution to currently-known inefficiencies in the copying of files from media that is sequential-access.
According to the preferred embodiments of the present invention, the storage component flowcharts are represented in
In step 502 of
In step 601 it is checked whether the storage component has previously begun retrieving files from a sequential-access volume and, if so, in step 602 the next file on that volume is selected. If a file spans into another sequential-access volume, the routine returns and the spanned-into volume becomes the current volume for the next file to be selected on that volume in the next routine run.
If a sequential volume is not currently being processed, in step 603 it is tested whether the method is already performing deferred processing of secondary volumes. If in step 604 it is determined that the file can be accessed on random-access media, it is immediately selected. Otherwise, if the file can be accessed on a primary sequential-access volume, that volume is selected from processing and the first file on that volume is selected in step 605. If the file is damaged, according to step 606, and can only be accessed on a secondary volume, that volume is placed on a list for deferred processing in step 607. Deferred processing of secondary volumes avoids thrashing that would be caused if a secondary volume were immediately mounted and used for transfer of files.
In step 608 for sequential copying from secondary volumes, when the inventory component has requested that residual (deferred) processing be performed, the next deferred secondary volume becomes the current volume and the first file on that volume is selected in step 605.
After performing the processing described above, the storage component returns to the inventory component for a new request.
Primary copies of the files are stored on either disk storage volume 807 or tape storage volumes 809 through 813. Secondary copies of the files are stored on separate tapes storage volume 815. In this example, as each file 805 in the inventory list 801 is requested, the method transfers the file from disk storage, if possible. If the file is stored only on sequential media, its corresponding volume is mounted and located such that the file may be accessed and copied. Therefore, to access file A, VOL2 809 is mounted and subsequently positioned to retrieve file A at the end of the tape. File B is requested next, requiring the same tape, VOL2 809, to be re-positioned such that file B may be copied from the beginning of that tape. File C is then copied from disk storage 807. File D is requested next, and requires the removal of VOL2 809, the mounting of VOL3 811, location of the first portion of file D at the end of the volume 811, removal of the volume 811 and mounting of VOL4 813, and location of the latter portion of file D on that volume 813. File E then requires removal of VOL4 813, re-mounting of VOL3 811, and re-positioning of VOL3 811 to the location of file E, which is found to be corrupted. At that point, secondary VOL5 815 would be mounted immediately for the retrieval of file E. File F requires VOL3 811 to be remounted and repositioned, and file G induces the removal of VOL3 811 and re-mounting of VOL4 813. Finally, file H is located by removing VOL4 813, re-mounting VOL3 811 and re-locating to the beginning of that tape. The lengthy volume mount order is listed at 817 in FIG. 8. The example detailed in this diagram is exemplary of the inefficiently inherent in the request and copy of copy data according to methods known in the prior art.
The present invention reduces this inefficiency by optimizing the retrieval of the same files on the same storage volumes utilizing an exemplary method of the invention. Referring to
The example as handled by the prior art, shown in
The inventory and storage component of the present invention are implemented in the present invention as software programs being part of the storage management server system of FIG. 1. These software programs can be stored on a storage medium for storing executable computer instructions, such as a magnetic diskette, an optical disk cartridge, or a magnetic tape cartridge, or in memories used to store digital representations of executable computer instructions, such as read-only memory (ROM) or programmable memory (PROM).
The foregoing description of the preferred embodiment of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. For example, the invention may be applied to the transfer of all files, and is not limited to only the most recent files or the generation of a copy set. Determination of position-optimal copying order may be in consideration of media format in addition to file position on the media. For example, consideration of file location on serpentine tape would be different than that for sequential tape, and would be done so as to minimize the number of tape passes. Another modification may involve the invention utilizing a network for file transfer between a client and a server, in contrast to utilization of removable media. Also, those skilled in the art will appreciate that the invention may be utilized with systems involving file aggregation, whereby all matching files would be transferred from within an aggregate once processing of that aggregate has begun. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5163148||Aug 11, 1989||Nov 10, 1992||Digital Equipment Corporation||File backup system for producing a backup copy of a file which may be updated during backup|
|US5263154||Apr 20, 1992||Nov 16, 1993||International Business Machines Corporation||Method and system for incremental time zero backup copying of data|
|US5276860||Dec 19, 1989||Jan 4, 1994||Epoch Systems, Inc.||Digital data processor with improved backup storage|
|US5276867||Dec 19, 1989||Jan 4, 1994||Epoch Systems, Inc.||Digital data storage system with improved data migration|
|US5403639 *||Sep 2, 1992||Apr 4, 1995||Storage Technology Corporation||File server having snapshot application data groups|
|US5404508||Dec 3, 1992||Apr 4, 1995||Unisys Corporation||Data base backup and recovery system and method|
|US5559991||Jun 27, 1995||Sep 24, 1996||Lucent Technologies Inc.||Incremental computer file backup using check words|
|US5642496||Sep 23, 1993||Jun 24, 1997||Kanfi; Arnon||Method of making a backup copy of a memory over a plurality of copying sessions|
|US5737763||Mar 22, 1996||Apr 7, 1998||International Computers Limited||Incremental disk backup|
|US6035379 *||Jan 9, 1997||Mar 7, 2000||Microsoft Corporation||Transaction processing for user data employing both logging and shadow copying|
|US6047294 *||Mar 31, 1998||Apr 4, 2000||Emc Corp||Logical restore from a physical backup in a computer storage system|
|US6078932 *||Jan 13, 1998||Jun 20, 2000||International Business Machines Corporation||Point-in-time backup utilizing multiple copy technologies|
|US6119131 *||Jun 12, 1998||Sep 12, 2000||Microsoft Corporation||Persistent volume mount points|
|US6148412 *||Sep 3, 1997||Nov 14, 2000||International Business Machines Corporation||Availability and recovery of files using copy storage pools|
|US6173291 *||Sep 26, 1997||Jan 9, 2001||Powerquest Corporation||Method and apparatus for recovering data from damaged or corrupted file storage media|
|US6286078 *||Sep 30, 1999||Sep 4, 2001||Storage Technology Corporation||System and method for allocating tape drives for multi-cartridge files|
|US6339778 *||Mar 24, 2000||Jan 15, 2002||International Business Machines Corporation||Method and article for apparatus for performing automated reconcile control in a virtual tape system|
|US6397308 *||Dec 31, 1998||May 28, 2002||Emc Corporation||Apparatus and method for differential backup and restoration of data in a computer storage system|
|EP0675490B1||Mar 28, 1995||May 17, 2000||Seagate Technology International||Method for reducing rotational latency in a disc drive|
|GB2323945A||Title not available|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7165083 *||Apr 30, 2004||Jan 16, 2007||Hitachi, Ltd.||File management method in a distributed storage system|
|US7254684||Mar 17, 2004||Aug 7, 2007||Hitachi, Ltd.||Data duplication control method|
|US7373393 *||Apr 2, 2004||May 13, 2008||Hitachi, Ltd.||File system|
|US7937546||Jun 28, 2007||May 3, 2011||Hitachi, Ltd.||Data duplication control method|
|US8234316 *||Jul 31, 2012||Microsoft Corporation||Nested file system support|
|US8499152 *||May 28, 2009||Jul 30, 2013||Trend Micro, Inc.||Data positioning and alerting system|
|US8782011 *||Mar 4, 2013||Jul 15, 2014||Symantec Corporation||System and method for scalable reference management in a deduplication based storage system|
|US9002807 *||Oct 10, 2012||Apr 7, 2015||International Business Machines Corporation||Generating enumerated information in which a plurality of files are enumerated in a sequential medium|
|US9280556||Feb 20, 2015||Mar 8, 2016||International Business Machines Corporation||Generating enumerated information in which a plurality of files are enumerated in a sequential medium|
|US20030101212 *||Nov 26, 2001||May 29, 2003||Borg Michael J.||Method for automatically completing an electronic form|
|US20050027718 *||Apr 30, 2004||Feb 3, 2005||Akihiko Sakaguchi||File management method in a distributed storage system|
|US20050108237 *||Apr 2, 2004||May 19, 2005||Hitachi, Ltd.||File system|
|US20050138309 *||Mar 17, 2004||Jun 23, 2005||Yuri Hiraiwa||Data duplication control method|
|US20050216523 *||Aug 2, 2004||Sep 29, 2005||Akihiko Sakaguchi||File management method in a distributed storage system|
|US20080005164 *||Jun 24, 2004||Jan 3, 2008||Terrence Yee||System and method for website development involving journaling and parent maps replacement|
|US20100082714 *||Sep 30, 2008||Apr 1, 2010||Microsoft Corporation||Nested file system support|
|US20110153968 *||Jun 23, 2011||Yuri Hiraiwa||Data duplication control method|
|US20130117239 *||Oct 10, 2012||May 9, 2013||International Business Machines Corporation||Generating Information with Plurality of Files Enumerated Therein|
|US20130185259 *||Mar 4, 2013||Jul 18, 2013||Symantec Corporation||System and method for scalable reference management in a deduplication based storage system|
|CN102165414B||Sep 23, 2009||Jul 23, 2014||微软公司||Nested file system support|
|U.S. Classification||1/1, 711/162, 707/999.204|
|International Classification||G06F3/06, G06F11/14, G06F12/16, G06F12/00|
|Cooperative Classification||Y10S707/99955, G06F3/0611, G06F3/0683, G06F3/0659, G06F3/0647, G06F11/1453, G06F2201/80, G06F11/1464, G06F11/1458|
|European Classification||G06F3/06A4T6, G06F3/06A4H2, G06F3/06A2P2, G06F3/06A6L4|
|May 15, 2000||AS||Assignment|
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CANNON, DAVID MAXWELL;DAWSON, COLIN SCOTT;MARTIN, HOWARDNEWTON;REEL/FRAME:010835/0403
Effective date: 20000515
|Aug 16, 2005||CC||Certificate of correction|
|Sep 23, 2008||FPAY||Fee payment|
Year of fee payment: 4
|Dec 10, 2012||REMI||Maintenance fee reminder mailed|
|Apr 25, 2013||FPAY||Fee payment|
Year of fee payment: 8
|Apr 25, 2013||SULP||Surcharge for late payment|
Year of fee payment: 7