Random ramblings

Saturday August 09, 2008

Wholesale DSL: Economics of Scenic Routing

In Ireland and the UK, due to the bulk of the POTS plant being owned and managed by the incumbent, former state telco, DSL tends to be provisioned on a wholesale model, where the incumbent telco is obliged through regulation to resell access to DSL customers to independent ISPs. The resale price is usually carefully regulated to be fair to both the incumbent and the market. The customer deals with only the independent ISP, who deals with the incumbent telco to arrange provisioning, etc. So the link layer (at least, immediate to the customer, independent ISP and incumbent) is something like:

customer1----telco exchange------<telco network>------ISP1
customer2----/ |                                ------ISP2
...            |
customern-----/                                 ------ISPn

The incumbent ISP tunnels the data traffic between the independent ISP and the customer (this is known as "backhaul"). So at the IP level, it looks like a single link (over which PPP is run), and the topology looks like (the customer numbers do not correspond with above):

customer1---<PPP>----ISP access----<ISP network>----<internet>
customer2---<PPP>---- router
..                    |
customern--<PPP>------/

This scheme must have management and implementation benefits, as it is a popular model. There is a reasonably clear layering, of IP and layer-2 and corresponding separation of responsibility. Maintenance and management of each layer is reasonably well decoupled, e.g. the telco can upgrade DSL head-end equipment at the exchange without having to technically interact with ISPs and the ISPs can carry out IP layer maintenance (addressing, whether to allow a PPP session, etc) without involving the telco. The only changes which routinely need interaction are the provisioning of a link between any specific customer and ISP (i.e. setting up a new customer, or changing the level of paid-for service) - to this end the telco can provide interfaces suitable for bulk updates as well as back-end<->back-end support[1].

So far... so what?

The downside to this model is that the IP topology is quite divorced from the geographic topology.[2] E.g. Imagine two DSL customers, each attached to the same telephone exchange (so they live reasonably close to each other). The IP topology between them, to which their packets are constrained if they want to communicate in some way via the internet, likely will be (at least, if we assume the ideal of competition that drove the creation of a regulated, wholesale DSL has paid off):

customer1-------ISP1------ISP2-----customer2

Because of how ISPs tend to interconnect, the chances are they'll do so at only a few locations, such as London, Amsterdam or Frankfurt (some Irish traffic might exchange at Dublin, not all though). So packets between customer1 and customer2 will wander off to some major, European metropolitan hub, before coming back to the exact same data-plane in the exact same networking device (no doubt saying a cordial hello as they pass their fellow packets still awaiting their long excursion). E.g.:

             +------------+
customer1----|  Exchange  |
customer2----| in Glasgow |
             +------------+
                   |
             <telco network>
               |       |
              ISP1    ISP2
               |       |
              LINX (London)

Such less-than-optimal routing is not uncommon in the networking world. Packets often pass close by packets of the same flow, before going off on a long detour, maybe because there is no direct route between two physically proximate devices, or even sometimes in the same device when the relationship of the flows is obscured by abstractions in the data-layers (IP over ATM, or MPLS, etc). This is because creating links and exchanging routes has a cost, particularly so in the ongoing management of a network. Physical links require technicians to setup and maintain. Logical links require network-administrators to configure. Exchanging routing information with other organisations requires further manual configuration (filters, policies) and monitoring, adding costs that further complicate the business-case evaluation and approval processes for new inter-ISP links. Further, for intra-AS forwarding, IP is perceived as hard to manage, and organisations often prefer to tunnel IP over an abstracted and flat topology, doing their network-engineering with protocol stacks like ATM or MPLS instead.

These costs mean a less-than-optimal route in a geographical sense will actually tend to be a uneconomic route, at least in private enterprise. Economics are the overriding metric, and so we should expect that routing tends to being economically optimal.

Yawn.. obvious.. So?

So one company owns and operates the DSL networking equipment, and potentially even the inter-exchange network. Typically this would result in a single IP topology, abstracted to some degree but still vaguely congruent with the physical topology, at least in the free market. In the wholesale-DSL model, the IP layer instead is provisioned primarily according to regulatory concerns. The telco is usually constrained from discriminating between ISPs, or giving itself preferential treatment where it also operates a consumer ISP. In the UK, the regulated, local-loop telco portion of BT is effectively a seperate company called BT Openreach; in Ireland, Eircom is supposed to maintain an internal chinese-wall between its wholesale and end-user operations. This distorts the economics somewhat.

E.g. often the telco itself operates an end-user ISP division. If it operated DSL head-ends as IP access routers[3], which is not too unreasonable to imagine given the example of some cable-ISPs, then at least the traffic of its own customers would stay local to the exchange. However, in the regulated, wholesale DSL model, this traffic MUST be "backhauled" to a few central locations, like the traffic of all other ISPs.

The only way the traffic could be kept local is if the telco/ISP could offer *all* ISPs the opportunity to exchange traffic locally, by the same method the telco's ISP uses. This would be difficult to do, as things stand, without unacceptable consequences.

Point please.. Or I must kill you

Ok, ok. The point, essentially, is that the benefits of competition for ISP customer-service comes at the cost of distorted economics, and IP topologies that are far less geographically optimal than those observed in single-organisation, end-user IP access networks (assuming economic efficiency isn't substantially different for both, so that the regulatory framework for DSL is the key variable). I.e.:

• The wholesale model results in a network that must cost more (and/or delivers less) than it would have otherwise, by simple comparison of IP topology alone.
• As the DSL model still results in a single, (effective) monopoly operator of the majority of the network[5] (at various levels[6]), any economic benefits likely lie primarily in competition in customer-service amongst ISPs.
• The attractiveness of multicast is reduced by the encapsulation of IP inherent to the wholesale-DSL model. If people from m different ISPs at an exchange subscribe to a multicast stream, the load on the trunk into the exchange is m, rather than 1. (Not that inter-domain, end-user multicast deployment has seen much use).
• Locality aware caching in P2P software (e.g. Joost) can not minimise load across paths that are hidden from IP (such as exchange trunks).
• It'd be interesting to read of studies on user-behaviour (traffic patterns) for MANs or NANs ("neighbour area networks", e.g. shared cable IP networks) to see if the locality makes any difference at all (with existing protocols), as well as studies analysing P2P traffic for temporal-locality to see what, if any, potential there is for significant trunk-link bandwidth savings, such as those for IPTV[4].

In short, my argument is that we're possibly in a bad place - regulating ourselves into a more costly system of delivering broadband, particularly for popular content, which becomes ever more difficult to substantively change the more the regulation encourages investment in it, which the private market likely will not be able to crack.

An Eircom executive recently, in a speech at the IETF plenary, gently argued that Ireland's DSL regulation was a disincentive to investment by Eircom (not reported on anywhere, it seems). So I'm somewhat hopeful that there's a still a chance for Ireland[7] to improve things, and reform broadband-access regulation. That's not to say we should give Eircom exactly what they want, only that they could be a driver to initiate reform, and of that itself we should not be sceptical..

With thanks to Thomas Bridge for insights and discussion on this subject (he very likely disagrees with my conclusions though).

See follow-up post on reform possibilities.

1. I can't think of a better term for "non-consumer support".

2. Obviously, this isn't new with customer/ISP internet access (e.g. dial-up, ISDN), however what is new with the arrangements for DSL is that the telco layer is increasingly (if not overwhelmingly these days) an IP-capable network, likely managed via IP (from the ADLS head-end equipment on), with parts of (if not all) of the DSL-data-delivery occuring over an IP-forwarding network (e.g. Eircom hand-off wholesale DSL as PPPoE/L2TP over IP). Ignoring the regulatory aspects..

3. Where IP capable - often the case with more modern equipment.

4. E.g. On Next-Generation Telco-Managed P2P TV Architectures, which found locality-aware caching could contain 80% of trafffic to within a DSLAM.

5. I'd be glad to hear of examples of where telco deregulation has lead to any significant access-link, "we own the wires" competition outside of juicy, major metropolitan areas..

6. The wholesale DSL model can result in mini-ISPs that run no more than DCANs (data-centre area networks), with the state telco connecting them to their customers AND their IP transit providers (perhaps unknowingly, for the latter, via more abstraction).

7. The UK, OTOH, seems to have a lot of ISPs who rely on BT wholesale DSL, so presumably that boat would be a lot harder to turn.

Thursday August 07, 2008

PGP: Signing policy update / Please sign my photo UID

Couple of PGP things:

• I've updated my signing policy.

  It's not terribly consistent though, and I'm still not quite sure where I stand on signing keys on nothing more than a cursory examination of governmental ID (i.e. people at key-signing events whom I don't know personally), so I'm not very happy with it. I'm much more comfortable signing keys of people I've known and interacted with over a period of time, and even more so when I know others who've done same - even if I've not seen an ID.

  Some may wonder why I would sign keys on the basis of my level-1/Low policy, but the concept of "web of trust" implies additive trust (potential for at least), so even a "not much confidence" signature ought to be of value to the WoT.

• I've added a photographic UID to my public key (yeah, horrible photo ).

  If you can attest to that being a good likeness of me and have good reason to believe it's my key, please do sign that new UID. (Also, what's with people who sign only one UID on a key? I've no control over the order of UIDs, I think, so its always a less favoured UID that gets signed in such cases - arg! ).

Weird BIND9 AXFR error? Remove stray A6 records..

Some release after BIND 9.2.1, its parser for A6 records appears to have broken. If you've ever experimented with A6, you might to go check you've expunged all occurrences from your zone files. Resultant symptoms include:

• Slave BIND servers fail to AXFR zone updates, reporting "transfer of ..... from .... : failed while receiving responses: unexpected end of input" in their log files.
• Manual AXFRs with dig gives the errors of ";; Warning: Message parser reports malformed message packet." and ";; communications error to ....: end of file".

Background: For many years, as I saw it only between a master behind a sub-1500 MTU (PPPoE, sigh) and certain servers, I assumed it was an MTU-blackhole problem with some upstream sneakily doing firewalling. However, recently the problem started to afflict another server, after an upgrade of their BIND software, and affected AXFRs even over paths that were perfectly fine. Some brute-force testing ruled out obvious problems like size issues (not like my zones are big) or relatively new records like SSHFP, leading to some head-scratching, but eventually pinned it down to a couple of stray A6 records.

Wednesday July 23, 2008

APNICs AS-dot policy proposal

In a similar vein to my attempt to dissuade RIPE from continuing with AS-dot format, James Spenceley has submitted APNIC policy proposal 65 to try get APNIC to change their ways.

Pretty Good BGP (PGBGP)

A new, interesting, soft/side-band approach to BGP security: Pretty Good BGP.

No crypto involved, no need for extensive deployment. Just monitoring the BGP routing table and reporting anomolous updates to the operator for further investigation. What's especially interesting is that they claim this system has discovered otherwise unknown hijacks of important prefixes.

Wednesday July 02, 2008

last.fm double plus good

Recipe for discovering new music:

• Go to last.fm and sign-up for an account
• Configure your MP3 player's last.fm plugin with your account details (you may need to download and install a plugin if your player does not already provide one - I use rhythmbox's).
• Listen to your own, favourite music for a while (e.g. a few weeks), to "scrobble" tracks to last.fm and seed your profile.
• Use your last.fm plugin's playlist features, such as "My Neighbourhood", "Artist Similar to" and/or "Tagged with" to discover new music.

When is a Sunni fundamentalist terrorist not a terrorist?

.... Why, when they're Baluchistani 'dissidents' agitating against the Iranian regime, of course! One could perhaps predict that in 20 odd years time Baluchistanis will be identified as being behind some terrible terrorist attrocity committed in the west, but then one would need a basic sense of history!

In other news, Nelson Mandela and other ANC members have been taken off the US terrorist watch lists. ANC terrorism of course was not terrorism, but instead the noble, indiscriminate maiming and killing of people who would have gladly volunteered to have died for freedom had they been offered a chance (and those who wouldn't have must have been apartheid lovers). Anyway, it seems the only way off the terrorist watch-list (besides being a US Senator or buddies with the head of the US DHS) is to be a successful terrorist..

In other news, scientists have discovered a new shade. Said to be neither white, nor black, it has tentatively been named 'grey' and even may possibly form a continuum. It is speculated the world may in fact be full of it...

Saturday May 31, 2008

Howto share your web-browsing experience...

• Get an account at a bookmark-aggregation site, e.g. ma.gnolia (del.icio.us if you prefer, but their HTTP RPC API seems to be unreliable).
• Add their "bookmark a site" smart-booklet to your browser's toolbar, and/or install their browser extension (GNOME's epiphany browser already supports ma.gnolia and del.iciou.us - install the epiphany-extensions stuff). Ma.gnolia have a nifty, web-2.0, AJAX bookmarklet thing that's quite cool.
• Link to your bookmark-aggregation account's feed from your site, so that those who are interested may subscribe to it.
• Consult the documentation of your bookmark-aggregation site to see if they have some handy javascript you can stick in the template of your blog that auto-includes your latest bookmarks in a side-bar (e.g. ma.gnolia do - see my blogs web site).
• If you have a social-network profile, check if a plugin is available to connect your bookmark-aggregation account together with it.
• Don't enable your bookmark-aggregation site's "post linkroll to blog" feature, especially not if you already know your blog is syndicated by blog aggregators (if you don't know, it probably is).

Really, despite what your ego says, your web-browsing habits are unlikely to be more interesting than the various mechanisms that your bookmark/tag aggregator already supplies for finding interesting links, which other people already use, cause link-rolls are tedious..

This was originally a comment on another blog, but either I forgot to hit "submit" or it got deleted.

Thursday May 22, 2008

The Security-Industrial Complex

This piece in the Rolling Stone magazine on China's surveillance state is well-worth a read.

This piece finally gives me a sense that perhaps I can understand the economic (i.e. corporate) motivations behind the ever-increasing dominance of security in civil life, something that has puzzled me for a while. For example, the military-industrial complex clearly is a factor* in the USA's use** of military-intervention as a foreign policy tool. However, I could never understand the bias that many western-governments have shown against freedom, in favour of encumbering us with, sometimes absurd, security measures. In my naivety, I thought it had something to do with some kind of "psychology of fear"-as-political-tool - not thinking of the security industry itself as being a significant driver.

This piece though puts it into perspective. Just as there are large industrial interests driving military spending, via a revolving door between the military, the government and the industry, so that piece makes it clear there are similarly large industrial interests, and a similar revolving door (except perhaps substitute police for military+) around security. This security-industrial complex is helping to drive the security policies of our western governments, and so cause growing amounts of public (or governmentally-mandated) spending to be sent their way. Given this encompasses spending on data-retention (private spending, by EU directive), national identity databases (UK), and so on, the amounts are not quite insignificant.

It's important to realise that security-service++ spending is at best of indeterminable utility+++. At worst, it may largely be wasted, other than to a small number of people who manage to make a lot of money for little work. Further, even if there is some utility to this security spending, that money may have achieved more had it been spent elsewhere, e.g. education, research, health-care, etc. Given the dubious utility of such security spending, the opportunity costs may well be far more significant than the amount of that spending itself.

Sadly I've little confidence anything is going to change in the near future. I'm vaguely hopeful though that eventually some nations will gain a competitive, economic advantage from foregoing massive-spending on security-theatrics and so influence other states. At least now, I think, I have slightly less naive understanding of it..

* In the sense that there is clearly a strong feedback loop in the USA of high military-spending sustaining significant military-industrial interests in the USA, which lobby to have the USA sustain its high level of military spending. The end-result is a state heavily invested in military power.

** Not that I claim that other nations are more enlightened. Just that most don't have anywhere near the same military dominance***.

*** I.e. I'm uncomfortable with militarism, not nations.

+ There's also a revolving door between the military and the police forces. Many coppers seem to be ex-armed-forces in the UK at least, though I don't have hard data.

++ Note that I say "service". Money spent on research, e.g. face-recognition, low-energy, x-ray tomography of passengers, might find other applications. An X-ray scanner operator though is not contributing as much to society, in that capacity, as they might otherwise. Also, I'm thinking mostly of the kind of additional spending on indiscriminate surveillance, data-mining and check-points prevalent since 9/11, rather than spending on more traditional security, policing and civilian intelligence.

+++ Exactly how dubious, no-one knows. There is this, though it talks of the opportunity costs of terrorism rather than just of security spending - related, but not quite the same (i.e. implicit in my opinion above is that the current security theatrics are essentially useless in terms of preventing any future terrorism). I don't know of the studies into this, would be interested to hear of more.

Monday May 19, 2008

How not to improve the security of your online banking website

• Implement a browser white-list

  While it may be a good idea to disallow customers with ancient, bug-riddled, phish-magnet browsers from accessing their online banking, doing so via a white-list of approved browsers is certain to annoy some of your more technically savvy users. Either implement a blacklist, so that the browsers you don't know about can still get in, or give the user the option to continue with an unapproved browser after warning them of the risks (it's their money..).

• Redirect HTTPS requests to your login page back to HTTP

  There's a good reason why your customers might use a local bookmark to a HTTPS URL to access their online banking. Don't defeat it by redirecting them back to HTTP! The fact that your customers only can usefully bookmark your frontpage, thanks to your use of weird URLs that redirect to stranger URLs for site navigation, is part of the problem - presuming you want to minimise HTTPS load.

This blog entry was inspired by the RBS web site

Monday May 12, 2008

The single-loop exception

Found myself having to act on a set of things, in some specific order. Certain items are exceptional and if present then processing stops there. The common idioms for this, that I've seen in C, are:

1. Tangled spaghetti-jungle of if/else with long conditionals
2. Goto break out

The former is common enough (though, not in your code nor mine, of course ) to make this blog posting worthwhile.

The latter is the neater approach, and possibly the only remaining legitimate use of goto today. However, it requires placing labels - which isn't error-proof - and maintaining discipline to not abuse (those labels are so tempting!). Some languages have dedicated syntax exception handling (try/throw/catch/finally), but these can feel a tad over-wrought for simple, localised exception handling.

There's another possibility though, generic to all C-like-syntax languages even, using a single-loop:

do {
  if (foo)
     do_stuff (foo);
  if (bar)
     break;
  if (acme)
     do_stuff (acme);
} while (0);

do_final_stuff();

The do {} while (0); pattern is of course already widely used in C, to encapsulate function-like macros. However, I've not personally seen it used in code bodies for such light-weight exception handling.

Another variant, that allows for some basic exception processing:

do {
  if (foo)
     do_stuff (foo);
  if (bar)
     break;
  if (acme)
     do_stuff (acme);
  return;
} while (0);

do_exceptional_stuff();

Wednesday February 06, 2008

USA taken offline by a cable cut!

Some other bad news that wasn't reported today: "I can't ping some-random-router.some-institute.edu - it used to work but it doesn't now. OMGZ DA USA IZ OFFLINE!!!".

That's bad news in the "Regurgitate ill-informed rumours" sense, simply cause a simplistic web-page says some host is down, which at least a few high-profile bloggers who should know better have fallen victim to (never mind news-aggregation sites who don't care what rubbish stories drive their impression count). Better and more fact-based commentary on how the cable-cuts have affected internet connectivity is out there..

Sunday January 27, 2008

Copyright in a digital era, how does that work?

If I see a programme in the EPG of my TV, on a "Freeview" channel (UK DVB-T digital telly channels), that I'd like to watch but can't at that time and if I then later instruct my computer to download that broadcasted programme via P2P, am I violating the copyrights of the rightsholders of that work?

(Imagine I am in a jurisdiction which recognises time-shifting as "fair-dealing" or "fair-use", as is the case in both the UK and Ireland. Imagine also that the downloaded programme will not be of appreciably better quality than the DVB-T MPEG stream my TV would have received. Does the answer change if I can not be certain the downloaded copy was made from the broadcast (e.g. if it was a film say, rather than a programme specific to the Freeview channel)?).

Thursday January 24, 2008

When all you've got is a spade..

You have to wonder if, on news that the Bush administration intends to spend the US out of a bad-debt-driven slump, the economists advising on this were cut off after ".. should work. Course, it just adds to our deficit, and encourages further spending on imports, so in the long term.."

Wednesday December 19, 2007

Wing-Suit Skydiving Videos

Disturbingly mad: Wing-suit mountain-road flyby. They seem to have really good aerodynamic control though - quite controlled, precise turns at least.

Apparently some in the skydiving community are hoping to eventually be able to land using just wing-suits (e.g. by landing on a slope, like a ski-jumper). To get an idea of the glide-ratio (and just how apparently insane these people are), have a look at this extremely low chute opening (again, note the precise turn-in to the landing zone).

There's a lot more of that stuff on Youtube. Completely nuts.