Forgive me, but I think this project is doomed.

The Data Protection Act is legislation governing the protection of personal data.

The datasets listed on data.gov.uk are not personal data; please see the FAQ.

There are quite a few datasets listed on data.gov.uk that go back more than ten years. For example this one:  Central Government carbon footprint, 1990 to 2008.

Danny

Owen is right in his repsonse.  You are talking about personal data, which would never be made available as Open Data in identifiable form, and also probably paper registered files.

Traditionally, paper files got shredded after five years unless someone made a case for their retention.  The main reason is obvious - sheer physical space - but if you kept everything in that environment, chance are you would be able to find nothing.  In theory, anything useful should have been retained, but given that reviewing archived files is time consuming (not to mention dusty) I would not be surprised if some paper got shredded that should not have been. 

The world started to change as e-mail and electronic storage became the norm.  These days, the majority of documents probably only exist electronically.   For a good few years, this played havoc with government records, because printing and filing paper copies tended to be low priority (and the filing clerks who would have got the job done were a thing of the past).  Records deteriorated into a mish mash of registered files, electronic stores and personal folders in your desk drawer.  This was chaotic, and, while mit took them far too long to do so, government departments had to tackle this, not least to be able to answer FOI questions.

These days, nearly all files are electronic too, so filing is easy, as long as the software is sufficiently user-friendly.  It would be naive to suppose that it always happens, particularly if the main job is taking all your time.  I am also aware that departments tend to delete e-mails in their staff's accounts after (say) a year, so anything unfiled at that point can just disappear).  I believe that there are still long term retention guidelines; that is, electronic files may be discarded after five years or so, but this is now more to do with getting rid of most of the chaff that hides the wheat rather than because of (electronic) space restrictions.  However, I would have thought that the easy option now was to say "retain", because it is so easy to pull up the file contents (or at least say you have!) rather than destroy (as the physical file had to come back from some distant old warehouse).

None of this direcly affects the availability of (non-personal and non-confidential) data, except in the sense that much more detail can be retained and made available in electronic form than in paper days (when probably only the end-result processed data would be available).  But I have a feeling that it is relevant to the metadata which help explain how the older processed data were obtained and processed.  Some of these metadata may well have met their maker.

I've described the real world of information handling as I recall it, rather than the theory of what should have happened.  Information management can go out of the window when under great pressure, and it would be unrealistic to suppose otherwise.  It is now given considerably greater priority by senior management than (say) fifteen years ago, but a cynic would observe that senior management tends to forget the time it takes to do it properly when assigning resources!  "I want the paper today" - OK, but if you always want that, when do you expect me to file it?