Bit Torrent file distribution at Sun
A while ago I put together a Bit Torrent solution for the OpenSolaris
folks. They wanted as many download options as possible for the
OpenSolaris bits. As is typical, we only had a couple of weeks to
select technologies and deploy a solution. Not a problem, I thought, I
had some familiarity with Bit Torrent so it seemed like it would be
pretty straight forward.
What we eventually decided on was a solution using a tracker called BNBT, and the Azureus bit torrent client. It all went together pretty easily however there were a few twists that we had to acaccommodatethe biggest of which was being able to restrict access to the torrents to users in non-embargoed countries.
For that, we decided on using a product called NetAcuity. The product has multiple language bindings including C and Java (which is great because BNBT is C++ and Azureus is written in Java). The software itself is fast and stable. The key was finding the right place in both BNBT and Azureus to interpose an IP Address check and permit or deny access.
The deployment environment presented a bit of a challenge as well. Anyone who's used Azureus knows that the main interface is a GUI, but GUIs aren't permitted in our hosting environment.. Its a pretty secure environment, Solaris is trimmed down considerably, including the removal of all of the X/Openwin/GNOME software.
However, after some plugging around on the Azureus Wiki, I found that an alternate TTY & Telnet interface was being worked on. And I had already known about the more established Swing WebUI. The TTY interface was fairly rudimentary at the time, but it was enough to allow it to be started and stopped without the need of an X server somewhere.
So that only left a model for torrent authoring. We devised a method that, although somewhat twisted, worked for low volume authoring. The method involved starting up a master or uber-seeder torrent client, Azureus, and remotely displaying it back to a desktop using the port forwarding capabilities of SSH. Within this master, the torrent file was created and the master started seeding the thtorrentThe torrent file would then get loaded by hand using the Swing WebUI into the seeders that we set up, there are four of those. After the torrent was downloaded and being seeded by all of our seeders, the master/uber-seeder could then be shut down until the next new torrent.
At the end of the day, it all has been working pretty well. At least for a first phase. But recently we started to look at how we could improve and streamline the framework. Particularly around torrent authoring.
So last week, we got together with a few folks from Aelitis, the main developer of Azureus. We described our environment and asked how it could be improved. They came back with a wealth of information on how to automate the process much better. There are some new features and UIs available that would help us greatly, including both an HTML WebUI and an RSS feeds. The method they suggested was to still create an master seeder, but enable a shared folder feature of Azureus which provides a way for it to automatically pick up new files in the shared folder and create torrents out of them. From there, the you would then point all the regular seeders to the RSS feed on the master seeder. They would periodically read the RSS feed, identify new torrents and automatically download and start seeding them.
Great stuff, that will go a long way to simplifying our torrent offering and provide a more scalable solution.
What we eventually decided on was a solution using a tracker called BNBT, and the Azureus bit torrent client. It all went together pretty easily however there were a few twists that we had to acaccommodatethe biggest of which was being able to restrict access to the torrents to users in non-embargoed countries.
For that, we decided on using a product called NetAcuity. The product has multiple language bindings including C and Java (which is great because BNBT is C++ and Azureus is written in Java). The software itself is fast and stable. The key was finding the right place in both BNBT and Azureus to interpose an IP Address check and permit or deny access.
The deployment environment presented a bit of a challenge as well. Anyone who's used Azureus knows that the main interface is a GUI, but GUIs aren't permitted in our hosting environment.. Its a pretty secure environment, Solaris is trimmed down considerably, including the removal of all of the X/Openwin/GNOME software.
However, after some plugging around on the Azureus Wiki, I found that an alternate TTY & Telnet interface was being worked on. And I had already known about the more established Swing WebUI. The TTY interface was fairly rudimentary at the time, but it was enough to allow it to be started and stopped without the need of an X server somewhere.
So that only left a model for torrent authoring. We devised a method that, although somewhat twisted, worked for low volume authoring. The method involved starting up a master or uber-seeder torrent client, Azureus, and remotely displaying it back to a desktop using the port forwarding capabilities of SSH. Within this master, the torrent file was created and the master started seeding the thtorrentThe torrent file would then get loaded by hand using the Swing WebUI into the seeders that we set up, there are four of those. After the torrent was downloaded and being seeded by all of our seeders, the master/uber-seeder could then be shut down until the next new torrent.
At the end of the day, it all has been working pretty well. At least for a first phase. But recently we started to look at how we could improve and streamline the framework. Particularly around torrent authoring.
So last week, we got together with a few folks from Aelitis, the main developer of Azureus. We described our environment and asked how it could be improved. They came back with a wealth of information on how to automate the process much better. There are some new features and UIs available that would help us greatly, including both an HTML WebUI and an RSS feeds. The method they suggested was to still create an master seeder, but enable a shared folder feature of Azureus which provides a way for it to automatically pick up new files in the shared folder and create torrents out of them. From there, the you would then point all the regular seeders to the RSS feed on the master seeder. They would periodically read the RSS feed, identify new torrents and automatically download and start seeding them.
Great stuff, that will go a long way to simplifying our torrent offering and provide a more scalable solution.
Posted by Ashish on May 03, 2006 at 10:41 PM PDT #