My colleague, Don O'Malley, asked me to post the following on resolving issues using 'wget' to automate patch downloads.  'wget' is a popular download method, and is used by patch automation tools such as 'pca'.

Summary: You can use versions 1.10.x and 1.11.x of 'wget' but not version 1.11.  Details of options to use are set out below.  See also Patch Download Automation using wget.

SunSolve recently migrated to using Akamai for patch and patch cluster downloads, to provide customers with a faster and more reliable experience.

Some customers have experienced issues accessing patches using 'wget'.  Here's information on the issues and how to resolve them:

1) You must use a version of 'wget' which supports 'https'.

Why?

SunSolve's new patch download service is accessed by redirecting requests to https://getupdates2.sun.com, which subsequently redirects to https://a248.e.akamai.net (Akamai).
Which versions of 'wget' support 'https'?
'wget' version 1.10.x or later has 'https' support.
How can I check which version of 'wget' I am using?
Run the command 'wget --version'

2) You must use the '-O' or '--output-document' switch in 'wget' to provide an output filename.

Why?

The Akamai URI identifying a patch is very long.  By default 'wget' will name the downloaded file the same as the URI.  As the filename is too long an error is thrown and the download will fail.
Example of the correct syntax:
# /usr/sfw/bin/wget ---http-user="xxxxxxxx" --http-passwd="xxxxxxx" --no-check-certificate "http://sunsolve.sun.com/pdownload.do?target=119255-01&method=h" -O /tmp/119255-01.zip

Example of some the output for a failing 'wget' request:

140778-01.zip?AuthParam=1251205908_479a27379ab5595128ae9170de4228c9&TUrl=L0QdUQV8Z4i0fdED3QTP3SJDWA8FMyaJsHfIWf4X29kTWQpKEzIbwqFuyRPZ&TicketId=3q3wk1CPNxhU&GroupName=SWUP&BHost=sdlc2h.sun.com&FilePath=%2Fpatches%2Fpatchroot%2Fall_unsigned%2F140778-01.zip&File=140778-01.zip: File name too long

Cannot write to `140778-01.zip? AuthParam=1251205908_479a27379ab5595128ae9170de4228c9&TUrl=L0QdUQV8Z4i0fdED3QTP3SJDWA8FMyaJsHfIWf4X29kTWQpKEzIbwqFuyRPZ&TicketId=3q3wk1CPNxhU&GroupName=SWUP&BHost=sdlc2h.sun.com&FilePath=%2Fpatches%2Fpatchroot%2Fall_unsigned%2F140778-01.zip&File=140778-01.zip' (Error 0).

3) If you are using 'wget' version 1.11.x you must use the '--auth-no-challenge' switch.

Why?

This is related to the manner in which 'wget' 1.11.x sends SunSolve a users Sun Online Account (SOA) information in this version of 'wget' (i.e. via '--http-user' & '--http-passwd'.)
Failure to include the '--auth-no-challenge' with 'wget' 1.11.x requests will result in the SunSolve Software License Agreement (SLA) being downloaded rather than the patch.
Example of the syntax for 'wget' 1.11.x users:
# /usr/sfw/bin/wget --auth-no-challenge --http-user="xxxxxxxx" --http-passwd="xxxxxxx" --no-check-certificate "http://sunsolve.sun.com/pdownload.do?target=119255-01&method=h" -O /tmp/119255-01.zip
Note, 'wget' version 1.11 does not have the '--auth-no-challenge' switch and so is not compatible with patch downloads from SunSolve.

4) You must provide 'wget' with direction on how to handle security certificate information.  Otherwise, patch downloads via 'wget' will fail.

Why?

Domains, getupdates2.sun.com & a248.e.akamai.net, are signed by trusted Certificate Authorities. (Verisign for Sun's and GTE Cybertrust for the case of Akamai.) Without a pointer to these certificates being provided to 'wget', download attempts will fail.
Which certs are required?
CN=GTE CyberTrust Global Root
CN=VeriSign Class 3 Secure Server CA - G2
What kind of error message can you expect to see from a failing 'wget' request?
ERROR: Certificate verification error for getupdates2.sun.com: self signed certificate in certificate chain
To connect to getupdates2.sun.com insecurely, use `--no-check-certificate'.
Unable to establish SSL connection.
Issue resolution:
If you wish to ignore this failure you can use the '--no-check-certificate' switch in 'wget'.  Example of the syntax:
# /usr/sfw/bin/wget --http-user="xxxxxxxx" --http-passwd="xxxxxxx" --no-check-certificate "http://sunsolve.sun.com/pdownload.do?target=119255-01&method=h" -O /tmp/119255-01.zip
If you wish to check against the certificates, you can use the '--ca-certificate' switch to point to a file containing the certificates.
http://sunsolve.sun.com/search/document.do?assetkey=1-9-240066-1 has an attachment called cacerts.pem, which is a concatenation of the two certificates.
If you save this file locally (eg to /tmp/cacerts.pem), you can use a syntax similar to:
# /usr/sfw/bin/wget --ca-certificate=/tmp/cacerts.pem --http-user="xxxxxxxx" --http-passwd="xxxxxxx" "http://sunsolve.sun.com/pdownload.pl?target=142284&method=h" -O /tmp/140778-01.zip

5) You may need to add firewall rules to enable 'wget' to work with SunSolve's new download service.

Why?

As the new download service is accessed by redirecting from http//:sunsolve.sun.com to https://getupdates2.sun.com initially and subsequently to https://a248.e.akamai.net, some customers may need to update their firewall rules to pass traffic from getupdates2.sun.com & a248.e.akamai.net in addition to sunsolve.sun.com.
How can I verify this?
Contact your System Administrator.

6) After associating a new contract to a SunSolve account there is a delay of up to 48 hours before 'wget' downloads will work for patches that the new contract should provide access to.

Additionally, customers registered in the Members Support Center must make an initial 'wget' call (which will fail) in order to trigger the synchronization process after associating a new contract to their party.

Why?

The delay is due to synchronization issues between SunSolve and the back-end access entitlement system.  Work is ongoing to reduce this delay.
What error message can you expect to see until this synchronization is complete ?
HTTP request sent, awaiting response... 403 You are not entitled to retrieve this content.

7) Attempts to download a patch README file by providing "method=r" in the URI is now failing.

Why?

Prior to the latest SunSolve release it was possible to download a patch's README file only via 'wget', using a syntax similar to :
# /usr/sfw/bin/wget --no-check-certificate --http-user="xxxxxxxx" --http-passwd="xxxxxxxx" "http://sunsolve.sun.com/pdownload.do?target=142284-01&method=r" -O /tmp/142284-01.README
There's a bug in the current SunSolve release this no longer works and attempts to download a patch README using this URI will result in a file of 0 Bytes being created.  This will be fixed at a later date.
Workaround:
Use "method=tr" to download a patch README file.  Example command syntax:
# /usr/sfw/bin/wget --no-check-certificate --http-user="xxxxxxxx" --http-passwd="xxxxxxxx" "http://sunsolve.sun.com/pdownload.do?target=142284-01&method=tr" -O /tmp/142284-01.README
Comments:

Let's do a thought experiment. If this tool was created by people who needed it to do their own daily work, what would it look like?

Posted by Mikael Gueck on August 27, 2009 at 01:10 AM IST #

What ever happenned to the KISS (Keep it simple, S..eriously) philosophy??.
Why waste CPU cycles on the server by encrypting files with https://?
And what the #### happened to good old FTP servers for crying out loud?

Sun is not alone in this behaviour.... IBM has copied you guys too. I came here after a Google search on that dreaded "self-signed certificate in chain" error message, after trying to download IBM's copycat office suite Lotus Symphony. Again, a convoluted process using https:// pages and authentication.

Control freaks have taken over the Net, I want to get off...

FC

Posted by Fernando Cassia on September 08, 2009 at 06:43 AM IST #

I'd be grateful for a correct syntax example for downloading recommended patch clusters. I'm having success with individual patches, but a request for 10_Recommended.zip fails. I have copied and rechecked syntax in a variety of wget examples and scripts.

Posted by Eugene Mosburg on September 22, 2009 at 07:46 PM IST #

Hi Eugene,

The following syntax will work to download Patch Clusters (I am downloading the Solaris 10 SPARC Recommended cluster in this example):

# /usr/sfw/bin/wget --no-check-certificate --http-user="SOA_USERNAME" --http-passwd="SOA_PASSWORD" "http://sunsolve.sun.com/pdownload.do?target=10_Recommended.zip" -O /tmp/10_Recommended.zip

There are a couple of points to note here:
* SOA_USERNAME & SOA_PASSWORD are your Sun Online Account username/password that you use to log onto SunSolve.
* All of the same requirements for wget, outlined in this blog (eg. https support and handling the certificate issues), apply to downloading patch clusters too.
* This syntax will ignore checking security certificates. Details of how to perform security certificate checking are included in the blog entry above.
* You need to have a support contract in order to be entitled to download patch clusters.
* If you want to download other patch clusters but are unsure of the URI to provide wget in place of "http://sunsolve.sun.com/pdownload.do?target=10_Recommended.zip":
- Go to the Patch Cluster page on SunSolve (http://sunsolve.sun.com/show.do?target=patches/patch-access) and find the cluster you want to download (eg. Solaris 10 x86 Sun Alert Patch Cluster).
- Right click on the "Download" link under "Download Cluster" and select "Copy Link Location" (in Firefox anyway!)
- Paste the new link into the wget query in place of "http://sunsolve.sun.com/pdownload.do?target=10_Recommended.zip" (eg. "http://sunsolve.sun.com/pdownload.do?target=10_x86_SunAlert_Patch_Cluster.zip" for the Solaris 10 x86 Sun Alert Patch Cluster)

HTH,
-Don

Posted by Don O'Malley on September 23, 2009 at 02:33 PM IST #

Thanks, Don! Your example works fine. I discovered my error, had been appending "&method=h" to the URL (which DOES work for download of indvidual patches).

Posted by Eugene Mosburg on September 24, 2009 at 03:49 PM IST #

Folks,
Is there a way to download just the "patch_order" file i.e. included with Recommended Patch Clusters? (using wget)
TIA
- SV

Posted by SV on September 29, 2009 at 04:35 AM IST #

Hi SV!

Paraphrasing from Ed Clark, the cluster author:

The Cluster README contains the list of patches in the same order as the patch_order file, so you could potentially cut-and-paste from there.

Having said that, it's not identical to the patch_order file, as the patch_order file contains adjacent duplicate entries for certain broken SWUP (Software Update Manager) patches, which need to be applied twice to ensure they apply correctly. The README patch order does not contain these duplicate entries.

Best Wishes,

Gerry.

Posted by Gerry Haskins on September 29, 2009 at 12:26 PM IST #

Hi SV,

I'm afraid you can't download the patch_order file without downloading the whole cluster.

The only resources that are made available for download are the complete cluster and the cluster README file.

That said, the patch cluster README file will contain a section entitled "PATCHES INCLUDED" that will list the patches included in the cluster in the order in which they should be installed (ignoring the duplicate patch issue Gerry has noted previously).
The cluster README file also contains a date at the beginning of the file, so that you can use it to determine whether or not the patch cluster has been updated.

You can use wget to download the cluster README file directly by using a syntax similar to:
# wget --no-check-certificate --http-user="SOS_USERNAME" --http-passwd="SOA_PASSWORD" "http://sunsolve.sun.com/pdownload.do?target=10_Recommended.README" -O /tmp/10_Recommended.README

HTH,
-Don

Posted by Don O'Malley on September 29, 2009 at 12:54 PM IST #

I keep getting the terms and conditions page when I try to retrieve a readme file. How do I get past that page with wget?

-regards

Posted by Mike on October 19, 2009 at 07:58 PM IST #

Hi Mike,

Can you confirm the version of wget that you are using by running the 'wget --version' command please?

If the version is 1.11.x, then please add the '--auth-no-challenge' option to your wget command to workaround being sent the SunSolve SLA, as opposed to the patch download you are requesting.

HTH,
-Don

Posted by Don O'Malley on October 20, 2009 at 12:07 AM IST #

Nice blog entry. I thought Sun had changed the privileges for wget users with no support contracts as my wget had suddenly stopped working.

I noticed a typo in your blog entry. In the main blog entry you specify to use:

--no-auth-challenge

As you indicated to Mike on Oct 20, 2009, the correct option is --auth-no-challenge. My clue was wget telling me that --no-auth-challenge was not a wget option.

wget works great once again :)

Posted by Sean O'Neill on November 01, 2009 at 05:12 PM GMT #

Thanks Sean!

Typo corrected.

Best Wishes,

Gerry.

Posted by Gerry Haskins on November 02, 2009 at 03:57 AM GMT #

wget fails to download recommended cluster but works fine with single patch.
(sanitized output below)

Cannot get recommended cluster with wget

# wget --version
GNU Wget 1.11.4

# wget --auth-no-challenge --no-check-certificate --http-user="myuser" --http-passwd="mypassword" "http://sunsolve.sun.com/pdownload.do?target=10_Recommended.zip" -O file.zip
--2009-11-09 17:45:44-- http://sunsolve.sun.com/pdownload.do?target=10_Recommended.zip
Resolving sunsolve.sun.com... 192.18.108.40
Connecting to sunsolve.sun.com|192.18.108.40|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2009-11-09 17:45:47 ERROR 403: Forbidden.

However, getting a single patch works perfect

# wget --auth-no-challenge --no-check-certificate --http-user="myuser" --http-passwd="mypasswd" "http://sunsolve.sun.com/pdownload.pl?target=110380&method=h" -O file.zip
--2009-11-09 17:49:46-- http://sunsolve.sun.com/pdownload.pl?target=110380&method=h
Resolving sunsolve.sun.com... 192.18.108.40
Connecting to sunsolve.sun.com|192.18.108.40|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://sunsolve.sun.com/pdownload.do?target=110380&method=h [following]
--2009-11-09 17:49:46-- http://sunsolve.sun.com/pdownload.do?target=110380&method=h
Reusing existing connection to sunsolve.sun.com:80.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://getupdates2.sun.com/all_unsigned/110380-06.zip [following]
--2009-11-09 17:49:50-- https://getupdates2.sun.com/all_unsigned/110380-06.zip
Resolving getupdates2.sun.com... 198.232.168.157
Connecting to getupdates2.sun.com|198.232.168.157|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://a248.e.akamai.net/f/248/21808/15m/sun.download.akamai.com/21808/patches/patchroot/all_unsigned/110380-06.zip?AuthParam=1257807114_8d852b5eddc498ca0e61a039cfaa&TUrl=L0QdUQV8Z4i0fdED3QTP3FMyaJsHfIWf4X29kTWQpKEzIbwqFuyRPZ&TicketId=OxpU7H6lA==&GroupName=SWUP&BHost=sdlc3i.sun.com&FilePath=/patches/patchroot/all_unsigned/110380-06.zip&File=110380-06.zip [following]
--2009-11-09 17:49:54-- https://a248.e.akamai.net/f/248/21808/15m/sun.download.akamai.com/21808/patches/patchroot/all_unsigned/110380-06.zip?AuthParam=1257807114_8d852b0f745eddc40e61a039cfaa&TUrl=L0QdUQV8Z4i0fdED3QTP3SJDWA8FMyaJsHfIWf4X29KEzIbwqFuyRPZ&TicketId=OxpU7kH6lA==&GroupName=SWUP&BHost=sdlc3i.sun.com&FilePath=/patches/patchroot/all_unsigned/110380-06.zip&File=110380-06.zip
Resolving a248.e.akamai.net... 204.8.50.33, 204.8.50.64
Connecting to a248.e.akamai.net|204.8.50.33|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 327240 (320K) [application/zip]
Saving to: `file.zip'

100%[==================================================================================>] 327,240 --.-K/s in 0.07s

2009-11-09 17:49:54 (4.62 MB/s) - `file.zip' saved [327240/327240]

#

Posted by Asif on November 09, 2009 at 11:04 PM GMT #

Hi Asif,

The error message that you are seeing when attempting to download 10_Recommended.zip - i.e.:
HTTP request sent, awaiting response... 403 Forbidden
2009-11-09 17:45:47 ERROR 403: Forbidden.
usually indicated that you are not entitled to retrieve the resource that you are requesting.

This is usually a result of either:
- Not having a valid support contract.
- Not associating your support contract with your Sun Online Account.

You can verify the contracts associated with your account by logging on to SunSolve and clicking on "Update Account" (http://sunsolve.sun.com/edit-user-form.do) on the top right of the page.
You will see your contracts listed in the "Current Contracts" field.

BTW - I suspect the reason that you can download patch 110380-06 without any issue is because that patch does not require a support contract.

HTH,
-Don

Posted by Don O'Malley on November 10, 2009 at 12:39 PM GMT #

Post a Comment:
  • HTML Syntax: NOT allowed

This blog copyright 2009 by Gerry Haskins