The complaint here is that, the Proxy's current cache lookup approach is too
"digital" - if a cache file exists, try to serve the client from the cache; else,
connect to the remote server. And the idea here is to Insert a new state in between
the two extreme states of "does not exist" and "exists". the new state, of course, is
"being created".
I mean, if the cachefile does not exist, check if the cache lockfile exists - which
would mean that some other thread is downloading the same URL at this very moment,
and so if we wait for a certian while, the download will be over, and we can pick
up the cache file.
Here is a script which does exactly that:
cache-root = "/somewhere/proxy-install/proxy-instance/cache"
cache-ndirs = 16
sleeptime = 100
#
# Generate the cachefile's name
#
cf = cachefilename $rq.reqpb.uri
#
# And "convert" it to the lockfile's name
#
lcf = $cf
lcf += ".lck"
if test -f $lcf
then
#
# The lockfile exists. What this means is, someone else is downloading
# this very same file at this moment. So let's just wait for the download
# to finish, and then pick up the cachefile. We save time, as well as
# network usage.
#
oldsize = 0
while true
do
millisleep $sleeptime
if test ! -f $lcf
then
#
# Ok, the lockfile has disappeared. So the download which
# we were waiting to end, is over. But just make sure that
# the new cachefile is in place.
#
if test -f $cf
then
#
# Cool - our "gamble" has payed off. Set the connect-mode
# to "never" so that we pick up the cachefile without
# attempting a connect to the remote server.
#
rq.vars.connect-mode = never
fi
break
fi
done
fi
#
# Over to the proxy-retrieve SAF now. It will find the cachefile in place,
# and since we have specified connect-mode = never, will serve the client
# from the cachefile.
#
req noaction
This script can be inserted as a Service SAF which gets executed just
before the proxy-retrieve SAF.
< Object ppath="http://.*">
ObjectType fn="cache-enable" query-maxlen="10" log-report="off"
ObjectType fn="cache-setting" lm-factor="0.10" max-uncheck="7200"
Service fn="nsh" script="/someplace/scripts/cachelookup.sh"
Service fn="proxy-retrieve" method="*"
< / Object>
Now say, we don't want this happening to all requests - maybe only to certian
requests, say those involving huge downloads, like mp3 files. In such cases, we can
just add something like this to the very beginning of the script:
if $rq.reqpb.uri != ".*.mp3"
then
req noaction
fi
