Utility Grids - Pods or Nodes directly connected to a core switch.
This is an interesting debate that has been going on for a while within the group, and in my view both can be right. It depends on the requirements as to which is correct for the use.
There are 2 main designs that have been around,
- 1) Pods - groups of compute nodes are connected to small switches and then brought back to a core switch,
- 2) All compute nodes connected directly to the core switch.
With all compute nodes connected to directly to a core switch you have a limit on the number of nodes that you can have in that grid because there are only so many ports on the core switch (this could be as many as 1200 depending on what switch you use), but once you have run out of ports and you need to expand the grid you then need to a number of new switches in order to cope with the interconnect that is required to keep within your maximum desired contention ratio.
With pods they can be specified to contain any number of compute nodes, again you need to look at the contention ratios but the core switch is not the blocking factor on how many nodes you can have in the grid. Each pod has its own set of compute and management switches that just need connections down in to the core but this is generally only 2 connection to each management and compute core for resiliency.
Pods are great as a modular system that allows growth in specific unit sizes. They need to be spec'ed to be the right size from the start, if they are too large then there is a lot of waste of nodes not being used, if they are too small then the core switch will run out of ports.
There is a limit on when it makes sense to have Pods as opposed to the nodes all connecting into a large core switch. Also it depends on the segregation required between Pods/Nodes which again comes back to the requirements of the grid.
If the requirement is for a large number of small grids < 30 nodes but will be reconfigured frequently then it makes more sense to have them connected to a core switch that can easily segregate between the nodes without physical access to them. This assumes that the total number of nodes does not exceed the port count on the core switch.
If the requirement is for a large number of nodes all configured into a single grid then the cost saving of getting cheaper switches and an aggregation switch to bring all of the smaller switches together then it makes more sense to have pods. Also the growth of the grid is not restricted to the number of ports on the core switch.
However with pods there are more network devices to manage which brings its own issues and proper management tools for grids are a must.
Posted at
03:59PM Nov 01, 2006
by Anthony Marshall in Work |