These are the simple steps to follow if you want to use your custom certificate for Web Services Security samples (from openssoproviders.zip) :
Note : Out of box, OpenSSO uses "test" certificate and its alias in bundled keystore.jks. The keystore or certificate store password and key password is "changeit".
1) Create custom certificate and custom keystore.jks using "keytool"
keytool -genkey -keyalg rsa -alias custom_cert_alias -dname "CN=custom.sso.apps.testbed, OU=Custom_org, O=Custom_company, L=Custom_city, ST=Custom_state, C=Custom_country" -keypass changeit -keystore keystore_custom.jks -storepass changeit
This creates keystore_custom.jks with "custom_cert_alias" certificate.
2) Replace server keystore
cd <opensso config dir>/<opensso_deploy_uri>
cp keystore_custom.jks keystore.jks
3) Replace client keystore
wherever you have unzipped openssoprovider.zip
.../resources/
cp keystore_custom.jks keystore.jks
4) Changed key alias from "test" to "custom_cert_alias" in Server -> Configuration -> Sites and Servers -> Default server configuration
5) Changed key alias from "test" to "custom_cert_alias" in Client <GF install dir>/addons/opensso/AMConfig.properties
Make sure keystore location is pointing to (3)
6) Changed key alias from "test" to "custom_cert_alias" in wsc, StockService (if you have this one), wsp agent profiles
Restart the GF container and you can exercise StockQuoteClient sample successfully with X509 token and signing on. Also verify that it is using custom certificate.