This chapter provides information to help you troubleshoot Directory Server and Directory Proxy Serverdata management problems.
This chapter contains the following sections:
This section describes how to troubleshoot LDAP operation failures. It describes the possible causes of the operation failures, the information to collect to help you troubleshoot the problem, and how to analyze this information.
An operation may fail for the following reasons:
ACIs are in place that do not allow the operation
Referrals are being followed to a different server
Updates can not proceed because a database has been set to referrals on updates
Database being reimported
Unallowed online configuration
To determine if ACIs are the source of your problem, gather information about all of the ACIs from the suffix level to the entry you are trying to access. Gather this data using the ldapsearch operation as follows:
# ldapsearch -D cn=Directory Manager -b base-suffix dn aci
Collect the access and errors log files that contain the operation. Be sure to enable the ACI logging level. Enable the ACI logging level for the errors log file as follows:
# dsconf set-log-prop errors level:err-acl
Enable the ACI logging level for the access log file as follows:
# dsconf set-log-prop access level:acc-internal
To view the contents of the error log, use the dsadm command as follows:
dsadm show-error-log -A duration [-L last-lines] install-path
The -A option specifies the maximum age of lines to be returned from the log. For example, to search for all entries younger than 24 hours, use -A 24h. The -L option specifies the number of lines to be returned from the log. For example, to return the last 50 lines, use -L 50. By default, 20 lines are returned.
To view the access log, use the dsadm command as follows:
dsadm show-access-log -A duration [-L last-lines] install-path
The log files themselves are located in the following directories:
If you are unable to troubleshoot your problem yourself, collect the error and access log files from the time during which the database was inaccessible and send them to Sun Support for analysis. By default, the log files are located in the instance-path/logs directory. To find the path to your error and access logs, use the following command:
# dsconf get-log-prop ERROR path
# dsconf get-log-prop ACCESS path
This section helps you troubleshoot when an SSL connection fails. It includes the following sections:
This chapter contains the following sections:
For information about troubleshooting SSL problems with Identity Synchronization for Windows, see Troubleshooting Problems With Identity Synchronization for Windows Over SSL
This section describes concepts to help you troubleshoot problems using SSL for Directory Server multi-master replication. Problems with SSL always appear on the supplier side. The error log will contain security related messages such as “SSL init failed.” or “Certificate not accepted.”
SSL connections always involve two participants:
The SSL client, which is the LDAP client sending the LDAP requests or the Directory Server sending the replication updates (the supplier).
The SSL server, which is the Directory Server accepting the LDAP requests (the consumer).
The SSL client initiates requests and the SSL server always receives the requests. During this exchange, the SSL server must provide credentials. Any SSL server needs to verify the credentials sent by the SSL client. In order to make this verification, the certificate database on the peer must contain the CA certificate of the certificate sent by the other peer.
In replication, SSL must be enabled in all replicas, even master replicas that only accept non-SSL operations. For example, a master server communicates with a hub server using SSL. The hub must listen on the SSL port. The master does not need listen on the SSL port because it is an SSL client. However, it must still define an SSL port, otherwise Directory Server can not initiate SSL certificate exchange for communication with the host server.
By default, SSL is enabled on all Directory Server 6.x instances. For a detailed explanation of how SSL works, see Secure Sockets Layer (SSL) in Sun Java System Directory Server Enterprise Edition 6.1 Reference.
Failure of an SSL connection could be the result of one of the following:
Wrong security libraries patch family applied
Server not configured to accept SSL
SSL port not open
CA certificate not found
CA certificate not appropriate or expired
SSL client not sending certificate when required
SSL server certificate not imported
This section provides information about collecting and analyzing data to help you troubleshoot SSL problem, including problems replicating over SSL.
The Mozilla website provides NSS Security Tools that are helpful for debugging and troubleshooting SSL problems. You can obtain the source-code of these tools from http://www.mozilla.org/projects/security/pki/nss/tools. This toolbox contains two tools, certutil and ssltap.
The certutil tool can be used to display all certificates stored in a certificate database and to display a single certificate in detail. Because it is possible to change or delete data in the certificates database when using this program, we recommend running the certutil tool on a copy of the original certificates database.
To use the certutil tool, you need to provide a password. However, the dsadm create command generates a default certificate database password that can not be retrieved. To use the certutil tool, change the certificate database password using the dsadm set-flags instance-path cert-pwd-prompt=on command.
The ssltap tool can capture the SSL communications between two systems. You must place the ssltap program between the connection from a Directory Server and an LDAP client. The program behaves like a Directory Server when it communicates with the LDAP client and behaves like the LDAP client when communicating with the Directory Server.
The certificates database resides instance-path/alias directory. Get the contents of this directory for each server involved in the problem.
For example, to see a list of the certificates that can be used as ns-slapd certificates (certificates with a u,, trust flags) use the dsadm command as follows:
dsadm list-certs instance-path
The command lists the certificates, such as defaultCert, the date from which it is valid, the date it expires, whether it is self-signed, who issued it, and to whom it is issued.
To see information about valid and trusted CA certificates (certificates with CT,, trust flags) use the dsadm command as follows:
dsadm list-certs --ca instance-path
This command provides the certificate alias, its dates of validity and expiration, whether it is built in, who issued it, and to whom it was issued. Verify that the SSL server and client certificates are generated by a certificate authorities that appear in the output of this command.
For detailed information about a particular certificate, use the dsadm command as follows:
dsadm show-cert instance-path certificate-alias
For example, the output of this command appears as follows:
server1 [/var/dsee/instances]> dsadm show-cert ds1 defaultCert Certificate: Data: Version: 3 (0x2) Serial Number: 00:85:8b:13:ef Signature Algorithm: PKCS #1 MD5 With RSA Encryption Issuer: "CN=server1,CN=Directory Server,O=example.com" Validity: Not Before: Fri Mar 23 14:10:51 2007 Not After : Sat Jun 23 14:10:51 2007 Subject: "CN=server1,CN=Directory Server,O=example.com" Subject Public Key Info: Public Key Algorithm: PKCS #1 RSA Encryption RSA Public Key: Modulus: 9a:c9:52:bd:ec:32:43:1a:39:96:90:02:f5:7e:18:45: 78:37:ca:8d:8f:c4:cc:6f:d1:7e:6c:38:d1:a1:53:41: 96:67:07:c7:c8:56:78:d1:f2:24:df:1f:eb:b2:07:5d: 6e:1f:58:fa:7a:f2:00:e4:95:d1:57:97:37:9d:22:31: 1c:b7:99:29:df:a3:8a:2a:87:e1:8b:54:ea:1f:7c:b7: 28:23:ce:be:7e:73:b3:87:f5:32:88:56:4e:58:68:f6: f6:01:2c:51:ca:07:00:40:ca:b3:9e:33:40:e8:f2:18: bc:16:d4:ac:ae:69:a7:c9:d7:g5:34:d4:87:11:2c:b1 Exponent: 65537 (0x10001) Signature Algorithm: PKCS #1 MD5 With RSA Encryption Signature: 29:76:4f:9f:ca:00:09:7b:05:ac:0f:26:6f:d1:93:aa: a8:c0:eb:a9:2a:39:e2:6e:08:0a:90:41:e5:7f:18:4a: 17:05:03:04:9b:ee:0a:dc:3c:ef:ee:aa:fc:ea:85:bf: f9:05:32:65:35:2c:e8:1f:32:9d:d6:a7:aa:68:a4:7a: e8:d9:4a:a0:a6:bc:fd:36:ba:d3:80:8a:1b:d3:81:8a: 68:1a:73:cc:36:7a:92:dc:eb:ec:af:02:6b:14:c7:77: e3:7d:95:19:e7:17:9d:d2:35:67:60:6b:9f:9b:d9:af: 01:f2:55:7f:5f:ce:23:a0:49:67:01:cd:30:38:8b:d2 Fingerprint (MD5): B8:34:27:AA:02:F6:07:FC:8F:D1:4A:AD:38:29:09 Fingerprint (SHA1): 3C:3B:BD:15:E8:1F:68:2E::E8:EJ:02:63:CD:8F:39:BE:DD:70 Certificate Trust Flags: SSL Flags: Valid CA Trusted CA User Trusted Client CA Email Flags: User Object Signing Flags: User
Confirm the validity of the certificate. Also, confirm that the issuer of the certificate is a valid and trusted certificate authority.
If you are using migrated 5.x instances of Directory Server, you can verify the contents of the certificates database using the output of the Certificate Database Tool, or certutil. The certutil tool displays the contents of the certificate and key database files. For more information about this tool, go to http://www.mozilla.org/projects/security/pki/nss/tools/certutil.html.
The certutil tool can be used by advanced users to populate the certificate database.
For example, run the certutil tool as follows:
# ./certutil -L -d /opt/SUNWdsee/alias -P slapd- Test (SUBCA1) Internal CA CT,C,C Test (CMSENG) Internal CA CT,C,C ESD SubCA1 Certificate u,,
The tool lists the certificates, such as Test (SUBCA2) Internal CA. and the trust flags associated with each certificates, such as CT,C,C. Very that the SSL server certificates are generated by a certificate authority that has a C,, flag. Verify that SSL client certificates are generated by a certificate authority with a T,, flag.
For example, you might have a certificate that works only as an SSL client, but you were trying to use it as a SSL server, which would not work. In replication, all Directory Server replicas need to have certificates signed by CT,, because they act as suppliers and consumers. Change the certificate trust flags to CT,, as follows:
# ./certutil -M -n cert-name -t CT,, -d /opt/SUNWdsee/alias -P slapd-
You can also run the certutil tool using the following options, to see the certificate authority that issued the certificate.
# ./certutil -L -n server-cert -d /opt/SUNWdsee/alias -P slapd-
Use this information to confirm that the certificate is present in the certificate database. You can also check the expiration date of the certificate to make sure that it has not expired.
You can configure client authentication to be required or allowed. Verify the setting client authentication settings by using the DSCC or by using the dsconf get-server-prop ssl-client-auth-mode command.
User's of migrated 5.x instances of Directory Server can verify the client authentication settings by checking the nsSSLClientAuth property in the dse.ldif file.
Go to the Directory Servers tab in the DSCC, and select the server from the table.
Click the Security tab and then the General tab.
In the Client Authentication section, go to LDAP Settings.
If you want only the SSL server to require the certificate, select Allow Certificate Based Client Authentication.
If you want both the SSL server and the SSL client to require a certificate, select Require Certificate Based Client Authentication.
Get a list of all the dynamically loaded libraries to see which NSS/SSL and NSPR libraries are being loaded. To get the list of dynamically loaded libraries on Solaris Intel, Linux, or Windows, used the following command:
# cd install-path /ds6/lib; ldd ns-slapd
To get the list of dynamically loaded libraries on Solaris SPARC, Solaris AMD64 or HPUX, use the following command:
# cd install-path/ds6/lib/64; ldd ns-slapd
The dynamically loaded libraries will be located in the following directory:
If you installed Directory Server using JES, verify that you have the correct libraries in the output of the pkginfo command.
You can use the ssltap tool to check if the hand shake is working on your system. The tool works like an SSL proxy, showing the communications between the LDAP client and the Directory Server and the packages being exchanged. For example, using this tool you might see where the server asks for a certificate but the client does not send the certificate or where the client proposes a cipher suite that the server does not support.
Since the SSL port 636 is hard-coded on the client side, the ssltap tool run on the Directory Server, where it must list on port 636 for incoming client requests. The SSL port of the Directory Server needs to be changed to a number other than 636 while running the ssltap tool.
For example, run ssltap as follows:
ssltap -vhfsxl -p 636 localhost:637 > output.html
After running some simple LDAP request on the client, such as ldaplist, the tool should have captures some SSL packets. Stop the tool by pressing CTRL-C and view the output file in a browser window. The output data is color coded so that data sent by the client is marked in blue and data sent by the server is marked in red.