Troubleshooting ISW deployments
The following is a troubleshooting guideline for Sun Java Identity Synchronization for Windows Installations.
1. Sun Java Message Queue – IMQ
#
/etc/init.d/imq start ( or /etc/init.d/stop to stop
the IMQ)
.
Now telnet to the port number of IMQ (TCP port 7676 is default). All the messages below should appear.
# telnet localhost 7676
Trying
127.0.0.1...
Connected to localhost.
Escape
character is '^]'.
101
isw-broker 3.6
portmapper
tcp PORTMAPPER 7676
admin tcp
ADMIN 17595
jms tcp
NORMAL 17594
ssljms
tls NORMAL 17596
cluster
tcp CLUSTER 17598
2. Sun Java Identity Synchronization for Windows – ISW
#
/etc/init.d/isw start (or stop to stop ISW)
Note that the port 7890 below is the connector port selected during the install of the DS Connector.
Telnet to the local port
# telnet
localhost 7890
Trying
127.0.0.1...
Connected to localhost.
Escape
character is '^]'.
®1v<?xml version="1.0" encoding="UTF-8"?>
<AuthRequestMessage>
<Parameter name="saint.msg.type"
value="AUTH_REQUEST"/>
<Parameter
name="saint.msg.auth.challenge"
value="1520478070584ee7fc8fff54b80e7be9"/>
<Parameter
name="saint.msg.auth.sessionKey"
value="wpCg7gdltYuk7HdS5DfZ8YUUObMdJDGVI46mqm9YHqvfKrQwXFprFg=="/>
<Parameter
name="saint.msg.requestID" value="0"/>
</AuthRequestMessage>
Netstat should reveal the processes running on port 7890.
# netstat -ad | grep 7890
*.7890 *.* 0 0 49152 0 LISTEN
hostname.27844 hostname.7890 49152
0 49152 0 ESTABLISHED
hostname.7890 hostname.27844 49152
0 49152 0 ESTABLISHED
hostname.7890 hostname2.company.com.31029 49640 0 49640 0 ESTABLISHED
In addition, a really useful script
that will show what UNIX processes are using particular TCP ports, and vice-versa. This script is useful for troubleshooting Identity Synchronization for Windows connectors. The Directory Server connector runs as a Java process listening on a particular TCP port. The ISW plugin in the Sun Directory Server connects to the connector over that port number.The script can be downloaded here
In this example, the Directory Server connector is listening on port 7890 and the corresponding Sun Directory Server is connected to the connector on port 7890
# sh
pcp -p 7890
PID Process Name and Port
_________________________________________________________
7493 /usr/java/bin/java 7890
DS Connector
sockname: AF_INET 0.0.0.0 port: 7890
sockname: AF_INET 10.200.131.36 port: 7890
sockname: AF_INET 10.200.131.36 port: 7890
_________________________________________________________
7766
/opt/SUNWdsee/ds6/lib/64/ns-slapd 7890 Directory Server
peername: AF_INET 10.200.131.36 port: 7890
3. ISW Logs and relevant entries
View logs in this directory and sub-directories
#cd /var/opt/SUNWisw/logs
These errors can be ignored
# tail /var/opt/SUNWisw/logs/CNN100/error.log
[10/Jan/2008:23:22:35.606 +0000] WARNING 14 CNN100 hostname.company.com "ElementGenerator: Can't find element 'Parameters' in element map"
# tail /var/opt/SUNWisw/logs/central/error.log
[10/Jan/2008:23:22:35.606 +0000] WARNING 14 CNN100 hostname.company.com "ElementGenerator: Can't
find element 'Parameters' in element map"
tail /var/opt/SUNWisw/logs/CNN101/error.log
[10/Jan/2008:18:01:13.491
+0000] INFO 10 "Log opened. Identity Synchronization for Windows build
2006.310.1625. Java runtime version is 1.5.0_09."
[10/Jan/2008:18:59:09.176
+0000] INFO 10 "Log opened. Identity Synchronization for Windows build
2006.310.1625. Java runtime version is 1.5.0_09."
If during an idsync resync, you get objectClass violation errors as follows then verify that the appropriate auxiliary, if any, objectClasses have been added in during the initial configuration.
# tail
/var/opt/SUNWisw/logs/central/error.log
[31/Jul/2008:18:36:43 +0000]
- ERROR<5897> - Schema - conn=-1 op=-1 msgId=-1 - User error: Entry
"uid=user001,ou=people,dc=company,dc=com",
attribute "gecos" is not allowed
[31/Jul/2008:17:00:52.680
+0000] SEVERE 33 CNN100 hostname.company.com "LDAP modify operation of entry
uid=user001,ou=people,dc=company,dc=com failed at null. Error code: 65, reason:
null" (Action ID=CNN101-11B7A05B0CA-274, SN=10)
Also verify that the hotfix for defect 6691600. (“Users with auxiliary objectclasses fail to link” has been applied – new connector.jar file). If the hotfix has been applied and you still get the errors, then make the following manual change
Execute the following ldapsearch command and look for the pswLinkAttributeRef:. entries. Make a note of the the corresponding “cn=???” value as below. (This value will vary on each installation.)
# ldapsearch
-h <hostnameOfConfigurationDirectory> -p <
dn:
cn=101,ou=Sun,ou=Globals,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswVersion: 4
pswUserObjectClass: inetOrgPerson
pswOtherObjectClass: companyUnixAccount
pswOtherObjectClass: shadowAccount
pswOtherObjectClass: posixAccount
pswNumOutboundConnectorThreads: 4
pswFlowInboundCreates: false
pswFlowInboundModifies: false
pswFlowInboundDeletes: false
pswFlowOutboundCreates: false
pswFlowOutboundModifies: true
pswFlowOutboundDeletes: false
pswCreatesAsModifies: false
pswModifiesAsCreates: false
pswPasswordAttributeName: userpassword
pswLocalRepositoryKey: nsuniqueid
pswRemoteRepositoryKey: dspswuserlink
pswSynchronizeDisables: false
pswPasswordKey: wW1dkRPGyapZIz2s+SvsAE9GYfyP0E2fczgfIIr1BhDyJZfhInr1xoNe12qBp/Yb
pswPasswordIV: goxvuULpkZpASxFBrjiD3tYf0E0hShbp
pswNoValueMeansEnabledInbound: true
pswOtherValuesMeanEnabledInbound: false
pswDisabledRoleRDN: cn=nsmanageddisabledrole
pswDisableMode: disabledRoleRDN
pswMetaAttributeDefaultRef:
cn=115,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswMetaAttributeDefaultRef:
cn=103,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswMetaAttributeDefaultRef:
cn=112,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswSignificantAttributeRef:
cn=105,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswCreationAttributeDefaultRef:
cn=106,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswCreationAttributeRef:
cn=113,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswCreationAttributeRef:
cn=104,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswCreationAttributeRef:
cn=107,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswCreationAttributeRef:
cn=114,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswCreationAttributeRef:
cn=109,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswCreationAttributeRef:
cn=102,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswHostsTopologyConfigurationRef:
cn=110,ou=TopoHosts,ou=Globals,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswSchemaLocationRef:
cn=110,ou=TopoHosts,ou=Globals,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswSignificantAttributeDefaultRef:
cn=116,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=c
om
pswLinkAttributeRef: cn=108,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
pswLinkAttributeRef: cn=112,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
cn: 101
objectClass: pswsundirectoryglobals
objectClass: top
For each of the blue entries in the prior search, search for the pswValue: dspswuser, entry as per below
#ldapsearch
h <hostnameOfConfigurationDirectory>
-p <
version: 1
dn: cn=108,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=Ide
ntitySynchronization,ou=Services,dc=company,dc=com
pswVersion: 4
pswName: objectclass
pswSyntax: 1.3.6.1.4.1.1466.115.121.1.15
pswValue:
dspswuser
pswValue: inetOrgPerson
pswPreferCreationAttributeDefaultToAction: true
cn: 108
objectClass: pswattributedescription
objectClass: top
Now, you have the DN required for modification (note that these portions of the DN may vary for your installation)
(cn=108,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=Ide
ntitySynchronization,ou=Services,dc=company,dc=com)
Stop ISW (/etc/init.d/isw stop)
Run ldapmodify against the ISW configuration Directory instance with the
following LDIF file:
(Again the DN value in
italics will vary for each installation).
dn: cn=108,ou=AttributeDescriptions,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
changetype: modify
replace: pswValue
pswValue: companyUnixAccount
pswValue: shadowAccount
pswValue: posixAccount
pswValue: dspswuser
pswValue: inetOrgPerson
Start ISW (/etc/init.d/isw start)
4. Workaround for defect 6709099. When installing ISW in an (multi-master replicated) MMR environment, the ISW plugins do not register with the ISW configuration Directory.
This is more of a cosmetic defect, since ISW may be functioning correctly but the status fails to show that the plugins are properly installed.
# sh
printstat.sh
Exploring status of
connectors, please wait...
Connector ID: CNN100
Type:
Sun Java(TM) System Directory
Manages:
dc=company,dc=com (ldap://hostname.company.com:389)
(ldap://hostname2.company.com:389)
State:
SYNCING
Installed on:
hostname.company.com
<ISW plugin information does not
display>
Connector ID: CNN101
Type:
Active Directory
Manages:
COMPANY.COM (ldap://ADDomainController.company.com:389)
State:
SYNCING
Installed on:
hostname.company.com
Sun Java(TM) System Message
Queue Status: Started
Checking
the System Manager status over the Sun Java(TM) System Message Queue.
System Manager Status: Started
There are two steps to complete:
- Ensure that the SUBC entries match between dse.ldif of the Sun Directory Server 6.3 servers and the configuration server
- Ensure that the plugin information is properly registered in the Sun Directory Server – Configuration Directory.
1. Ensure that the
SUBC entries match
Proceed as follows.
View the dse.ldif file for the Sun Directory 6.x server that has the plugin installed. The entry for the ISW plugin in the dse.ldif contains the name of the plugin as defined in the configuration Directory; highlighted below:
dn: cn=config,cn=pswsync,cn=plugins,cn=config
objectClass: extensibleObject
objectClass: top
accessorhost: hostname.company.com
accessorport: 7890
accessoruser: gEQ8OUyzepyAX1lw
accessorpassword: nhyji5OQxlCyfvuOLyqwm1jmewwSDxA6Sdm4lWeG7dI=
subcomponentid: SUBC101
subcomponentsaintssloption: false
debugloglevel: INFO
cn: config
creatorsName: cn=directory manager
entrydn: cn=config,cn=pswsync,cn=plugins,cn=config
createTimestamp: 20080718192517Z
auditloglevel: FINE
modifiersName: cn=pswsync,cn=plugins,cn=config
modifyTimestamp: 20080728205759Z
The corresponding entry in the Configuration Directory must contain the identical information. If not make corrections as needed. See screenshot below.
Note: that the DN of the corresponding entry in the configuration Directory Server will vary per installation. In the examples below the DNs are:
cn=139,ou=SyncHosts,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
cn=17,ou=SyncHosts,cn=active[4],ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
The cn=active[4] component of the DN will vary for each installation.
Similary for any additional 6.x Directory Servers, make corrections as required. The additional Directory Server below has an entry of SUBC101.
2. Ensure that the plugins are registered
The second modification required is as follows:
The configuration Directory must be be configured for each plugin. In the screenshot below, notice that pswInstalledSubComponentInfo contains <none>. This is incorrect.
Modify the entry so that pswInstalledSubComponentInfo contains the names of each plugin as follows:
pswInstalledSubcomponentInfo: SUBC100?ldap://hostname.company.com:389
pswInstalledSubcomponentInfo: SUBC101?ldap://hostname2.company.com:389
Important, the SUBC100 number must match SUBC100 in the modification 1. above.(ensure subc entries match)
Note that there are two DNs to be modified:
cn=active_as_ADP101,ou=Status,ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
cn=active_as_ADP10,ou=Status,ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
Now the entry appears as follows:
And executing printstat shows that the plugins are installed.
# sh
printstat.sh
Exploring status of
connectors, please wait...
Connector ID: CNN100
Type:
Sun Java(TM) System Directory
Manages:
dc=company,dc=com (ldap://hostname.company.com:389)
(ldap://hostname2.company.com:389)
State:
SYNCING
Installed on:
hostname.company.com
Plugin SUBC100 is installed on ldap://hostname22.company.com:389
Plugin SUBC101 is installed on ldap://hostname.company.com:389
Connector ID: CNN101
Type:
Active Directory
Manages:
.company.com (ldap://ADDomainController.company.com:389)
State:
SYNCING
Installed on:
hostname.company.com
Sun Java(TM) System Message
Queue Status: Started
Checking
the System Manager status over the Sun Java(TM) System Message Queue.
System Manager Status: Started
An alternative to using the DirectoryServer 5.2 console to register the plugins, is to execute ldapmodify at the command line with LDIF files similar to this: (Similar because the DN and SUBC numbers will vary)
dn:
cn=active_as_ADP100,ou=Status,ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
changetype:modify
replace:pswInstalledSubComponentInfo
pswInstalledSubcomponentInfo::SUBC?ldap://hostname.company.com:389
pswInstalledSubComponemtInfo::SUBC101?ldap://hostname2.company.com:389
Posted at 02:46PM Jul 31, 2008 by Jonathan Gershater in Identity & Directory Server | Comments[1]
Thursday Jul 31, 2008















PLEASE STOP AND READ THESE COMMENTS!!!!!!!!!!!!
Corrections to this Blog post are the following:
Section: 4. Workaround for defect 6709099. When installing ISW in an (multi-master replicated) MMR environment, the ISW plugins do not register with the ISW configuration Directory.
SubTopic: 2. Ensure that the plugins are registered
INCORRECT:
Note that there are two DNs to be modified:
cn=active_as_ADP101,ou=Status,ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
cn=active_as_ADP10,ou=Status,ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
CORRECTION: Only ONE DN is required to be modified.
You FIRST need to make sure the ISW adapter your going to alter is the Active Directory. In most installs it is ADP101 (second adapter installed).
You can search the ISW configuration (ou=services) for the DN which starts with, "cn=ADP101,ou=Adapters" and verify the "pswDirectorySourceRef" attribute value contains
"ou=ActiveDirectory".
The DN to verify and change is:
cn=active_as_ADP101,ou=Status,ou=GlobalConfig,ou=1.1,ou=IdentitySynchronization,ou=Services,dc=company,dc=com
You need to make sure it contains only ONE pswInstalledSubcomponentInfo attribute with the value of "pswInstalledSubcomponentInfo:: PG5vbmU+"
Note: The value is in base64, make sure to use this EXACT value. Decoded this value is "<none>".
** NOTE **: ALWAYS backup your ISW configuration to an LDIF file PRIOR to any manual modifications you perform, as performing these manually can render your ISW invalid and could require you to perform a full reinstall.
Posted by Joel Moran on October 21, 2009 at 12:55 PM PDT #