You Want To Store What? For How Long?
I can't resist sharing this recent story with you from IpSecurityWatch.com titled "Video Storage To Exceed 3.3 Exabytes by 2012". I've been around the computing environment for a long time (how long I have stopped admitting but suffice it to say in my younger days I was keypunching and loading tapes on a mainframe to run jobs). I never thought I'd see the day when it would be predicted that a particular industry application (video surveillance) would require exabytes worth of storage. For that matter, I remember the days when exabyte wasn't a word yet (or, maybe it wasn't used much then).
Enough about age...let's talk video storage...being an employee of Sun Microsystems, it doesn't take a rocket scientest to figure out that this type of headline would catch my attention (and that of many of my colleagues). And, going beyond the headline to consider what this really means stirs up quite a debate. Just today a group of us were discussing this topic and there were varying views. One: the majority of surveillance data is worthless and doesn't need to be stored. Two: but, who is to judge what 'worthless' means? And, does time factor into that too? It may seem worthless at the time that an incident occurs (a child wanders off from a public playground) but what about later - if that child is reported missing and days later that video at the public playground might provide some clues?
I just came from a meeting with a team working on an RFP for a state entity that is moving to all IP video surveillance - quickly. And, they require storage of the video data for 365 days. You can imagine the discussion that started in that meeting. I'm still hoping to find solid evidence of a mandate or policies establishing requirements for video surveillance data archival. I hear references all over the board and for all kinds of reasons. The police chief at a large city in the U.S. says they will be keeping data for 7-10 years because of the Freedom of Information Act yet I haven't found a direct reference to storing video surveillance data. I've heard of a port authority in Europe requiring 20 years of storage for their video surveillance data, yet a colleague in the U.K. tells me there are restrictions on the amount of time video surveillance data can be stored - something like a few days, I think.
The bottom line in all of this, to me, is whether people coming from the security side of this discussion really understand the ramifications of storing data of this nature (video) for long periods of time. There are questions related to accessibility (how often and how quickly do you need access to the archived data), the cost of storing data on disk (very, very costly - especially compared to long term storage options such as computer tape), the security of that data...just some things to consider.
Now, a shameless plug for Sun's approach to video surveillance and the lifecycle of the video data. There is a new data sheet on our website titled "Sun Open Video Surveillance Data Sheet" (download it from the right hand column) that takes you through Sun's three-tiered approach to storing video surveillance data. This approach is designed to address the issues I raised above related to accessibility, cost control and data integrity.
Enjoy and have a great weekend!
Hi Shereen,
This topic is actually very apposite for me, given that I've had yet another enquiry in the last few days, regarding discussion of the technical fixes available for reducing large scale video data retention.
The problem is actually very straightforward, insofar as all the various people concerned each have their own answers to the questions as to how much information should be recorded, and for how long should it be retained?
So, starting at one end of the chain, the DVR / NVR manufacturers have one set of answers based on their marketing objectives, the installer / integrator has another view, the customer invariably wants to spend as little as possible, the law enforcement agencies have there objectives to be met, and then finally the criminal justice / courts services have their unique requirements, just to add a little spice to the mix.
The problem in the real world is that like a chinese whisper, what starts off as best intentions at one end of the chain, eventually results in a recording structure which barely meets the most basic of requirements for those agencies much further upstream.
For example, if a customer (the security manager) wants to record for just seven days (as an example), the police may prefer 30 - 60 days retention to permit post event investigations to be completed, and the legal eagles could require all data recorded at or around the days pre and post event, to be retained for many months or even years.
The problem is compounded by peripheral techniques, such as Video Motion Detection (VMD) or Video Analytics which although designed to help reduce the amount of data being recorded, can actually compromise the admissibility of those sections which have been retained.
Here in the U.K., the "Data Protection Act" which is the main piece of legislation that governs most CCTV recording, actually allows for recordings to be retained for an appropriate period, which is generally taken to mean around 30 days. Whilst this is pretty much standard practice now across the country, in reality, a much longer retention period would be preferred by many investigating officers working with organised crime and Homeland Security related cases.
Is there a definitive solution? .... well possibly not, but that's not to say a workable solution cannot be adopted as "best practice". Will it happen anytime soon? .... we live in hope :-))
Posted by Jon on August 29, 2008 at 06:14 PM MDT #