Best Practices in Corporate Archiving
"At the centre of all archiving thinking is the 'record'. Whether a parchment court roll, a frontier-land patent, a business report on a paper file, or an electronic message, all records have three properties: content, structure and context."
"The biggest single worry federal agencies have in the records management field today is how to capture e-mail records. The agencies recognize that e-mail constitutes their greatest source of liability in the event of litigation, FOIA requests, congressional investigation, or media exposure."
1. Introduction
The quotations above illustrate the changing environment for the management of records and archives. Because of the way records are created and managed in the electronic environment archives can no longer be managed in isolation from records. However, although the technology has changed, the basic requirements for managing records remain the same. It is a matter of ensuring that record keeping systems are designed and implemented to provide the controls to guarantee integrity of the content, structure and context of records.
In the past, records were viewed as having a life cycle with distinct stages. At the end of their active life, records were appraised and their disposition status determined. They could be destroyed or preserved for the longer term. The guardianship of active records in an organization was usually the responsibility of the records office, possibly under the control of a records manager. Long-term preservation decisions were the work of archivists. However, this perspective has changed. An important and influential new model for records management places records in a continuum, which includes the additional stages of creation and disposition. Records managers and archivists are partners in this new environment, together with IT personnel, corporate information managers, web masters and end users who all play a role in managing records and archives throughout the continuum.
From the evidence of the past few years, mistakes are clearly being made in the management of corporate records, and some examples are described in this paper. In many organizations, accurate, reliable and trustworthy records that fulfill the necessary requirements for evidence are not being created and properly managed. There are, however, identifiable strategies, processes and procedures, which are key elements in this new approach for ensuring best practice records management. This paper will review the steps for the design and implementation of a best practice corporate record keeping system.
2. Background
2.1 The origins of modern records and archives management
The problems developing from increasing output of electronic information are not new. They were described in 1989 by David Bearman in a UN Report, which discussed the need for revised policy and guidelines to manage computer generated records, including electronic mail. This report noted that electronic records policies needed to change and that records should be evaluated "according to the function they perform and support and not on the media they are created and reside on." This assertion defined an important perspective and firmly linked the management of records and archives.
In the early 1990s Cox, Bearman and their colleagues at the University of Pittsburgh carried out an extensive research project, the prime objective of which was to identify the functional requirements for the preservation of electronic evidence. Bearman noted, "One must identify the requirements for record keeping in each community and then establish metadata that meet those requirements." It is metadata that carries the information about the context and structure of the record, and it is metadata that can carry the record forward in space and time and facilitate the integration of archives and records management.
2.2 Archiving 'as it was' in the world of paper records management
In the paper world of the past, records created in the course of business were stored and kept in hard copy form as evidence of an action, decision or process. Secretaries and filing clerks who maintained records systems with filing cabinets containing the paper evidence of an organization's business largely ensured corporate record keeping. Typically, at the end of the period of usefulness or active life of these records, an archivist's services would be employed to appraise the records and determine which of them was worthy of long-term preservation and which should be disposed. This approach is still in practice in many organizations. It remains a fact that, although many records are 'being born electronically,' paper is still viewed as the official record. It is also a fact that not many of these e-records are stored in an electronic records management system or on file in hard copy format.
2.3 Growth of digital output
Technological change has resulted in the exponential growth of digital output in a wide range of formats. According to the School of Information Management and Systems at the University of California, Berkeley, the world produces between 1 and 2 exabytes of unique information per year, which is roughly 250 megabytes for every man, woman and child on earth. An exabyte is a billion gigabytes.
Ninety-three percent of the information produced each year is stored in digital form. Magnetic storage is by far the largest medium for storing information and is the most rapidly growing, with shipped hard drive capacity doubling every year. From the perspective of records management the key factors are:
1. Printed documents of all kinds comprise only .003% of the total;
2. Individuals create and store a large amount of the unique information. 55% of hard drives are installed in single-user desktop computers;
3. Digital content dominates and is the largest form and the fastest growing. Most textual documents are now ‘born digital;'
4. Email is growing very rapidly, and this includes corporate email. 610 billion email messages are sent each year worldwide, and 2.2 billion messages are sent each day.
2.4 Consequences of inadequate record keeping?
A number of consequences follow from poor record keeping. For instance, because of the large volume of information being created, there may be system paralysis or, at the very least, hindrance in accessing information. There are almost certainly costs associated with the purchase of additional storage. The lack of a systematic approach increases the risk of wholesale, unsystematic and possibly illegal destruction, which in turn leads to the loss of valuable business and archival records. An improperly managed system carries with it the risk of security breaches and the unauthorized alteration or deletion of records resulting in possible loss of evidence and public embarrassment. There are more than a few, high-profile examples of this.
2.3 Examples of records management failures
There are an increasing number of law cases involving electronic evidence. Potentially damaging information can often be found on old backup tapes, in emails, and on disks. When lawyers decide they need electronic evidence as part of the 'discovery' process, they usually approached the IT department of a company and ask them to investigate. Trying to discover the evidence "can become an economic and operational nightmare." This alone is a compelling reason for having an electronic document and records management system that can store records in such a way that they can be retrieved easily. The fact that an organization does not know what records it is retaining points to a lack of developed records practices and procedures. If the records are being inadvertently preserved, then the organization is incurring unnecessary costs for storage and possibly litigation risk if there is incriminating evidence contained within.
E-mail is a major source of corporate memory and evidence of business processes, and it leaves a "pervasive trail of evidence that demonstrates corporate decisions and behavior – digital testimony to an organization's functions, activities and transactions." As such, email trails have been found to be an invaluable weapon in providing evidence of who decided what, when and why.
The following examples of records management, or rather mismanagement, illustrate why best practice archiving and records management is essential to an organization. What the examples most clearly demonstrate is that there is a need for written document retention policies for email.
Microsoft and evidence of anti competitive practices found in email
Microsoft Corporation has been in the spotlight more than once in terms of finding damaging evidence in emails, and it is Bill Gates who has been in the hot seat. For example, a January 1999 email surfaced in a court case in 2002 in which Gates described a plan to use the Windows operating system to promote Microsoft's audio and video delivery software over that of rival, RealNetworks.
RM Comment: In the Microsoft case, best practice archiving with effective retention policies may have resulted in the destruction of the message long before the case was heard. Well-defined policies and procedures may also have prevented the comments from ever being committed to paper or exchanged by email. For some information, email is an inappropriate medium, but an organization needs a policy in which this is clearly outlined.
Merrill Lynch and the "piece of junk" evidence
This case occurred in October 2000. A well-respected analyst at Merrill Lynch Analysts, Henry Blodget, was recommending the stock of a company called Infospace. At the same time, he wrote an email to a colleague describing the stock as "a piece of junk." As it transpired, the message became public as a result of a subpoena by New York Attorney General Eliot Spitzer, who contended that Merrill Lynch was deceiving investors in order to gain investment banking business. Merrill Lynch did not acknowledge wrongdoing but paid out $100 million to settle the case.
RM comment: As a result of this case, Merrill Lynch has recently begun a mandatory firm-wide, email training program. Unfortunately, this will be unlikely to address the email management problem. The reason is that evidence of business should be appraised and managed according to record keeping requirements. Policies cannot just be written for email but "need to be referenced to the 'business functional areas' they deal with and that this need's to be reflected in email use and retention policy instead of treating email as an undifferentiated mass or as a single documentary form that was called ‘email'."
Enron, Arthur Andersen and incriminating e-mail
Accounting firm Arthur Andersen was convicted of deliberately destroying documents to thwart an investigation being carried out by the Securities and Exchange Commission. Evidence emerged that the instruction to shred documents had been given to and carried out by Duncan, an Andersen employee and lead partner on the Enron account. Arthur Anderson asserted that Duncan had acted independently in this regard, but this was shown to be untrue based upon evidence of an altered memo provided to the court. Arthur Andersen, and not Duncan, was found guilty of obstruction of justice.
RM comment: Even if retention policies had existed at Arthur Andersen, they would likely not have been followed as the documents were destroyed knowing that an investigation was pending. However, having a well defined, understood and implemented records management system in existence may have resulted in a more serious approach to the importance of the organization's documents and may have prevented the employee from undertaking the shredding. Subsequent to this case more stringent punishments have been imposed upon companies which undertake document shredding in this kind of situation. The new Sarbanes-Oxley Act makes destroying or attempting to destroy documents related to a federal investigation a crime punishable by up to 20 years in jail. The conviction of Arthur Andersen on a charge of obstruction of justice showed that "the act of destroying evidence in anticipation of a lawsuit can lead a jury to the conclusion that the information would have been damning".
Prudential Insurance and the "lost order"
In 1995, pending the review of a claim against the company, the presiding judge issued an order requiring Prudential Insurance to preserve all documents potentially relevant to the litigation. However, the order was apparently not passed on to the employees, and relevant documents were destroyed.
RM comment: There was no evidence that the document destruction was deliberate, but the loss of the documents did remove potentially valuable evidence for the claimants. The judge held senior management responsible for the loss and fined Prudential $1 million. In addition he ordered the company to inform employees of the original court order (to retain documents) and to submit to the court within 30 days, a written manual describing its document retention policy.
2.5 The problems and how to deal with them
The cases above illustrate a variety of problems that would not occur, or would be a lot less likely to occur, if a system of best practice corporate record keeping and archiving were in place. Such a system involves establishing the necessary processes and procedures to meet the records requirements of an organization.
In order to implement the requirements, it is important to choose an electronic document and records management system that can provide the tools to manage records in accordance with the standards of best practice. This will minimize risk, reduce costs (such as storage and retrieval in the event of legal 'discovery' cases) and increase business efficiency by ensuring access to the corporate memory.
An electronic document and records management system should provide:
Facility to apply document retention policies: Based upon the above cases, the system must include the functionality to easily apply and enact written retention policies. The cases above reinforce the necessity for document retention policies that are well thought out, and complied with, throughout an organization. While such policies will not necessarily prevent all the problems, they will certainly reduce confusion and provide more trustworthy evidence that a company has been acting professionally and in accordance with well-established practices.
Easy access from the Desktop: To ensure that records are committed to the electronic records system, there must be easy access from the user's desktop. This requires a platform that can provide out-of-the-box integration with an organization's standard applications such as email, word processing etc. It must be possible to catalog records seamlessly into the system with the appropriate metadata.
Trained users and an information management structure with defined policies and responsibilities: The end user must know what constitutes a record, what constitutes evidence. In the case of email, perhaps this is even more difficult. Rick Barry recently related his experiences consulting with organizations on their email use and their policies to oversee that use. Based on surveys and follow-up focus groups with organizational staff members, Barry found that: up to 80% of email creators stated that they did not "have a clue" as to when their email qualified as an official record, that there was "great inconsistency" as to what was actually classified as a record, that staff were "largely unaware" of the existence of organizational email policies and that they had an unwarranted fear that saved email would come back to haunt them. And finally, they believed that despite claims that email was being routinely copied to the records center, very few actually were.
Deletion and destruction capabilities: In the conventional paper-based world, once a document is shredded, incinerated, or buried in a landfill, it is no longer subject to discovery as a practical matter. However, the routine "deletion" of a computer-based document does not destroy the data. Hitting the "delete" key merely renames the file in the computer, making it available for overwriting if that particular space on the computer's hard disk is needed in the future. The data may remain on the hard disk or on removable storage media for months or years, or may be overwritten only partially. A viable electronic records management system should include a shredder delete function which enables an administrative user to overwrite and make a record completely irretrievable, should it be required as part of policy.
Comprehensive metadata: Metadata in electronic systems is vital to the control of records. A valuable EDRMS provides extensive metadata in line with International standards and enables the kind of control of records that is essential to preserve their integrity over time. The role of metadata in describing records and their content, context and structure is very significant. For paper records, the metadata is often implicit. That is to say, the structure of the record does not need to be defined as it is apparent, the content is largely specified (with perhaps the addition of more indexing) and the context of the paper record usually depends heavily upon its physical location in the filing structure. In the case of electronic systems however, metadata must be created either automatically or by data input. Metadata is used to record the processes of capture, registration, classification, access and security classifications, identification of disposition status, storage, use and tracking and implementation of disposition. These processes all have detailed specifications that the electronic records system must be capable of meeting.
Classification tools: Email, in the same way as other records, must be part of the business classification plan. A good EDRMS provides the tools to build a business classification scheme that can represent the business functions of an organization. The classification or file plan enables records to be better organized, described and linked. This in turn provides easier access, better retrieval and enhanced use and distribution capacities. A thesaurus provides the opportunity to use terms that have specified and hierarchical relationships with one another. A hierarchically arranged thesaurus should conform to the standards defined in ISO 2788. Vocabulary controls involve the use of a list of authorized subject headings that may be used in the classification scheme. The list allows control of the terminology used to name records. Disposition authorities control the decisions of what records are generated and for how long they will be retained. Retention is determined according to such things as the legal and administrative requirements for maintaining records, the uses for the record (short and long term), links with other systems and other stakeholder's interests.
3. Working towards best practice corporate archiving
The International Standard for Records Management ISO 15489 was published in 2001. ISO 15489 reflects the perspective of an integrated regime, or continuum, for the management of records and archives. A key strength of the standard is its recognition that records are evidence of an organization's activity and their management is critical to business interests. ISO 15489-1 identifies the essential elements of records management, and ISO15489-2 describes guidelines for implementation of the Standard.
Section 3 of part two, the Standard Guidelines, describes the 'Design and Implementation of Records System' (DIRS) process. This is a methodology in eight steps for the design and implementation of a records system. The methodology applies to records in any format and addresses how to design, develop and implement a records system that complies with ISO 15489. Below is a description of the steps involved:
Step One is the preliminary investigation. This step is essential groundwork, as it is the process during which the organization identifies the major factors of the environment in which the business operates. Factors to consider are the administrative, legal, business and social contexts. The analysis can be used, for example, in preparing a business case for records management. It can also define more clearly the role that records management plays in the corporate culture.
Step Two, the analysis of business activity, examines in detail what the organization does and what are its main functions and activities. The analysis starts with the big picture view of the organization, its functions, and drills down to the smaller parts, the activities within functions and resultant transactions. This functional model for records provides a conceptual view of the organization and the framework for the development of control tools essential to the electronic records system. These control tools include the business classification scheme, thesaurus, disposition authorities and security and access schemes.
Step Three, identification of requirements for records, is the step that provides the basis for design of the system. The impact of regulatory, business and community requirements are identified and evaluated in relation to the creation of business records. A risk analysis is a necessary part of this step. These requirements can change over time, so this process is one that is revisited periodically in accordance with changes that may have an impact. For example, there may be new legislation that affects record keeping practices e.g. the Sarbanes Oxley Act, the corporate structure may change, or additional committees may be established for which there are new record keeping requirements.
Step Four, the assessment of existing systems, measures the extent to which business activities are documented within the present information and record keeping systems. A records inventory assists in this process. Also needed are the details of existing record keeping practices and the extent to which they fulfill the requirements for creating evidence. The results of this assessment are useful in defining the specification for a new electronic records system, such as what integrations may be required.
Step Five is the identification of strategies for satisfying records requirements. To identify the strategies required a thorough knowledge of the organization is needed. This follows from the analyses done in the previous steps. There are four main strategies or tactics to consider. These are policy, design, implementation, and standards:
Policy strategy
Policies need to spell out what an organization intends to do and may include a program and procedures to show how the policies may be implemented.
Design strategy
Design applies to the technical components of the record keeping system. For example, one important objective may be to make the records creation as seamlessly integrated with business processes as possible. Another requirement may be that IT specialists, records managers and archivists work together in designing a system to meet requirements.
Implementation strategy
Implementation includes managing the configuration of hardware and software and the establishment of procedures and practices for record keeping, e.g. how email is captured and managed. It can also include training, which is a powerful implementation tactic.
Standards strategy
Standards are key elements of best practice. As a result of the rapid increase in digital information, the last few years have witnessed a significant growth in the number that are applicable to the records management environment. Depending on the kinds of records that are kept, different standards are taken into consideration. Some examples of relevant standards are:
· DoD 5015.2 Design Criteria Standard for Electronic Records Management Software Applications.
This standard sets forth mandatory and optional baseline functional requirements for Records Management Application (RMA) software. The standard includes a requirement for email filing, retention scheduling and event-driven disposition.
· The Dublin Core standard was developed by the Dublin Core Metadata Initiative and is the most widely used standard for resource discovery.
· The Recordkeeping Metadata Standard for Commonwealth Agencies (RKMS) defines a basic set of metadata elements and 65 sub-elements for best practice record keeping for Australian government records.
· The Victorian Electronic Records Strategy (VERS) addresses the management and long-term preservation of electronic records in the Victorian (Australia) public sector.
· MoReq: This is a specification sponsored by the European Commission and describes the characteristics of an Electronic Records Management system (ERMS) and how ERMS data should map to physical document stores.
In Step Six, the design of a records system, new technologies are taken into account and plans are made as to how the necessary changes are incorporated into the existing records management structure. It is important at this stage to consult with all those involved in Records Management including RM professionals, IT personnel, the Chief Information Officer, Webmaster etc. Paper and electronic records require different processes and controls.
Step Seven, implementation of a records system, includes a project plan showing in detail how implementation takes place and the steps involved. These will likely include the conversion and migration process, integration plans, training, documented policies, procedures and standards, performance reports and reports to management. At the end of this stage the system is in use and improved records management practices should be in place.
Step Eight, post-implementation review, is the final stage. A system of evaluation and ongoing monitoring regime are put in place including surveys, interviews with management and other stakeholders and checking operations. This enables an organization to establish more clearly the return on its investment and also to keep track of how the system is meeting changes to records and organizational needs.
Conclusion
Best practice corporate archiving depends upon an understanding and a philosophy throughout the organization that defines records and archives within an integrated process of records management. This new regime for records and archives requires the recognition that the management of records relies upon the combined efforts, expertise and actions of many people within an organization. This includes records managers, archivists, information technology specialists, CEOs and end users.
The evidence abounds that many organizations lack the necessary processes and procedures for managing electronic records. This leads to business inefficiency, increased costs and exposure to unnecessary risk. Many organizations are using outdated records management practices that no longer apply in the digital environment. Ensuring the reliability and authenticity of records requires a system that can record and preserve, in the metadata of records, the content, context and structure of records over time.
The DIRS process described in ISO 15489 provides a methodology that describes how to implement a best practice records system. While it almost certainly involves choosing a suitable software tool, it also requires a careful and thorough analysis of an organization's culture, information management structure and business processes. Only after this analysis is complete, is it possible to plan, design and implement the changes that are necessary for a modern regime of records and archives management.