The following documents the logs that are created by a fairly normal company and the related time that these logs should be maintained.
Security and audit logs and the reporting on the systems that produce these logs must be centralised. It is necessary to have these records centralised so that systematic attacks across the servers can be pinpointed in a timely fashion. It also significantly lowers the risk of the logs being compromised.
The security logs must be backed up and kept for the duration of the audit period (e.g. 1 Year minimum). This can be extended with other requirements and should be deemed to be the absolute minimum requirement.
For instance, in Victoria, changes to the Crimes Act (1958) [Crimes (Document Destruction) Act 2006; Act No. 6/2006] have created “a new offence in relation to the destruction of a document or other thing that is, or is reasonably likely to be, required as evidence in a legal proceeding”. This act, punishable by indictment for a term of up to five years imprisonment affects anyone who destroys or authorises the destruction of any document that may be used in a legal proceeding (including potential future legal proceedings).
The distinction between correct and circumstantial evidence is that direct evidence categorically establishes the fact. Circumstantial evidence on the other hand is only suggestive of the fact. Authentication logs are generally accepted as direct evidence short of proof that another party used the access account. The consequence of this is that system logs can be critical to a legal case and hence must be maintained.
A section noting the “minimum document retention guidelines” has been included below.
- “Can you tell me when Joe Bloggs logged in and logged out?”
- “Can you see if someone has tried to open this file”
- “Can you tell me who the last person to open this file was?”
- “We need to know if this person has been chatting (Office Communicator) with this person outside the company”
- “We need to know what these people have been chatting about (office communicator) both inside and outside the company”
- “We need to know when people add external contacts to their communicator list”
- “Who had this IP address on
- The concern is leaking of information, more people resigning and things of that nature. What can you give me?
In addition to normal system logs, an operating log must be maintained that records any significant events and action taken by the system operator or administrator. These logs should be maintained on a centralised and isolated server. Proper recording would indicate whether operators were following instructions for halts in programs, change control, etc.
There are more reasons to maintain logs than for security alone. Some of the reasons to maintain system and network logging include:
- Optimizing system and network performance; to record the actions of users
- Identifying security incidents, policy violations, fraudulent activities, and operational problems;
- Performing audits and forensic analyses;
- Supporting internal investigations;
- Establishing baselines; and
- Identifying operational trends and long-term problems.
The following should be used as guideline in storing logs. This table sets the times for local storage. The central logging system should maintain all logs for at least 12 months online and maintain an offline backup for longer periods.
|Category||Low Impact Systems||Moderate Impact Systems||High Impact Systems|
|How long to retain log data?||1 to 2 months||3 to 6 months||12 to 36 months|
|How often to rotate logs||Optional (if performed, at least every week or every 25 MB)||Every 6 to 24 hours, or every 2 to 5 MB||Every 15 to 60 minutes, or every 0.5 to 1.0 MB|
|Transferring log data to the log management infrastructure - how frequently that should be done?||Every 3 to 24 hours||Every 15 to 60 minutes|| At least every 5 minutes |
(If not automatic – e.g. syslog-ng)
|How often log data needs to be analysed (through automated or manual means)||Every 7 days||Every 12 to 24 hours||At least 6 times a day|
|Whether log file integrity checking needs to be performed for rotated logs||Optional||Yes||Yes|
|Whether rotated logs need to be encrypted||Optional||Optional||Yes|
|Whether log data transfers to the log management infrastructure need to be encrypted or performed on a separate logging network||Optional||Yes, if feasible||Yes|
The analysis of the logs should be automated and use a correlation system to make review simpler.
Entries from the logging system that are deemed to be of particular interest should be retained on the system and also transmitted to the log management infrastructure. Reasons for having the logs in both locations include the following:
· If either the system or infrastructure logging should fail, the other should still have the log data. For example, if a log server fails or a network failure prevents logging hosts from contacting it, logging to the system helps to ensure that the log data is not lost.
· During an incident on a system, the system’s logs might be altered or destroyed by attackers; however, usually the attacker will not have any access to the infrastructure logs. Incident response staff can use the data from the infrastructure logs; also, they can compare the infrastructure and system logs to determine what data was changed or removed, which may indicate what the attacker wanted to conceal.
System or security administrators for a particular system are often responsible for analysing its logs, but not for analysing its log data on infrastructure log servers. accordingly, the system logs need to contain all data of interest to the system-level administrators.
All logs could (should) be sent to a SQL database. Syslog, Syslog-ng and many other logging formats can be sent to MySQL, Informix and other database formats. This provides the capability to create custom queries to a database in order to retrieve selected logs.
· Anti-malware Software. The most common form of antimalware software is antivirus software, which typically records all instances of detected malware, file and system disinfection attempts, and file quarantines. Additionally, antivirus software might also record when malware scans were performed and when antivirus signature or software updates occurred. Antispyware software and other types of antimalware software (e.g., rootkit detectors) are also common sources of security information.
· Intrusion Detection and Intrusion Prevention Systems. Intrusion detection and intrusion prevention systems record detailed information on suspicious behaviour and detected attacks, as well as any actions intrusion prevention systems performed to stop malicious activity in progress. Some intrusion detection systems, such as file integrity checking software, run periodically instead of continuously, so they generate log entries in batches instead of on an ongoing basis. This section includes Tripwire and a NIDS (such as Cisco Secure IDS or SNORT).
· Remote Access Software. Remote access is often granted and secured through virtual private networking (VPN). VPN systems typically log successful and failed login attempts, as well as the dates and times each user connected and disconnected, and the amount of data sent and received in each user session. VPN systems that support granular access control, such as many Secure Sockets Layer (SSL) VPNs, may log detailed information about the use of resources.
· Web Proxies. Web proxies are intermediate hosts through which Web sites are accessed. Web proxies make Web page requests on behalf of users, and they cache copies of retrieved Web pages to make additional accesses to those pages more efficient. Web proxies can also be used to restrict Web access and to add a layer of protection between Web clients and Web servers. Web proxies can keep a record of all URLs accessed through them. This would include the ISA systems and other Web Filtering systems.
· Vulnerability Management Software. Vulnerability management software, which includes patch management software and vulnerability assessment software, typically logs the patch installation history and vulnerability status of each host, which includes known vulnerabilities and missing software updates. Vulnerability management software may also record additional information about hosts’ configurations. Vulnerability management software typically runs occasionally, not continuously, and is likely to generate large batches of log entries.
· Authentication Servers. Authentication servers, including directory servers and single sign-on servers, typically log each authentication attempt, including its origin, username, success or failure, and date and time. This would include the Security and Application events from the Active Directory DNS servers.
· Routers and Switches. Routers may be configured to permit or block certain types of network traffic based on a policy. Routers that block traffic are usually configured to log only the most basic characteristics of blocked activity (as is the case generally). These logs should be increased to incorporate both allowed and dropped traffic. Switch traffic should log unusual events associated with VLANs and any violations of ACLs.
· Firewalls. Like routers, firewalls permit or block activity based on a policy; however, firewalls use much more sophisticated methods to examine network traffic. Firewalls can also track the state of network traffic and perform content inspection. Firewalls tend to have more complex policies and generate more detailed logs of activity than routers. In many companies, logging is disabled for both allowed and blocked traffic rendering the logs of little use. The Firewall logging needs to be changed to record both ALLOWED and DENIED Traffic to and from the network.
- DNS (All active servers)
- DHCP (All active servers)
- Network Devices
- Firewalls (Cisco ASA, Load Balancers)
- Cisco Router and Switch Logging (Both internal and external switches)
- SAMBA logging (Unix/Linux SMB Servers)
- Web server logs (Apache, IIS, TomCat, PHP etc)
- Proxy Logs (Websense, ISA)
- Tripwire logs (Servers)
- Mail Server logs (Sendmail, Exchange etc)
- Databases (MySQL, Informix, MSSQL)
· System Events. System events are operational actions performed by OS components, such as shutting down the system or starting a service. Typically, failed events and the most significant successful events are logged, but many OSs permit administrators to specify which types of events will be logged. The details logged for each event also vary widely; each event is usually time-stamped, and other supporting information could include event, status, and error codes; service name; and user or system account associated with an event.
· Audit Records. Audit records contain security event information such as successful and failed authentication attempts, file accesses, security policy changes, account changes (e.g., account creation and deletion, account privilege assignment), and use of privileges. OSs typically permit system administrators to specify which types of events should be audited and whether successful and/or failed attempts to perform certain actions should be logged. The Windows Domain systems should be configured to log security events.
- · AV logging (Malware detected etc)
- · Deep Freeze logs(and similar products)
- · Authentication logs (from Domain Controllers)
|Basic Commercial Contracts||6 years after discharge or completion||4 years after discharge or completion||6 years after discharge or completion|
|Deeds||12 years after discharge||A minimum of 6 years after discharge||12 years after discharge|
|Land contracts||12 years after discharge||6 years after discharge||12 years after discharge|
|Product liability||A minimum of 7 years||Permanent||A minimum of 10 years|
|Patent deeds||20 years||25 years||20 years|
|Trade marks||Life of trade mark plus 6 years||Life of trade mark plus 25 years||Life of trade mark plus 6 years|
|Copyright||75 years after author’s death||120 years after author’s death||50 years after author’s death|
|Contracts and agreements (government construction, partnership, employment, labour, etc.)||A minimum of 6 years||Permanent||A minimum of 7 years|
|Capital stock and bond records||7 years after discharge||Permanent||12 years after discharge|
Any logging from systems associated with the above listed areas should be maintained for the period listed as a minimum.
“Logging mechanisms and the ability to track user activities are critical in preventing, detecting, or minimizing the impact of a data compromise. The presence of logs in all environments allows thorough tracking, alerting, and analysis when something does go wrong. Determining the cause of a compromise is very difficult without system activity logs.”
This section of the PCI requirements then sets out the formats required for logging.