Authority Documents

The URL to request that document from the API.

This is either a 1 or a 0. It indicates whether the record is live within the database, or should be redacted.

Because the UCF™ treats every ID as both unique and persistent, we never delete an ID once used, nor do we re-use the ID. Therefore, if we have to redact a record, we merely mark the Live Status as moving from 1 (live) to 0 (redacted).

All records are initially created and marked by the system as Live (1). There are certain scripts that the UCF’s database team will run to ensure that two instances of automated deprecation takes place:

1. If an Authority Document has been deprecated, all of its citations will be deprecated.

2. If a control has no citations pointing to it, the control in question will be deprecated.

Other than the instances noted above, records must be deprecated as an editorial process and approved by both the editorial reviewer and the editorial approver. When the Live Status is set to deprecated (0), there might also be a corresponding setting for the Deprecated By element, but this is not mandatory.

If a record in the UCF needs to be deprecated, the record will not be deleted from the system. Instead, the record will be marked as deprecated (its "Live Status" field will be set to 0), and the Deprecated By field will be filled out with the ID(s) of the record(s) that took its place (if any).

Initially this element is blank and only a UCF editorial process can indicate a Deprecated By content change. That change is then reviewed by the editorial reviewer and editorial approver. If there are contents in this field, the Live Status field must be set to deprecated (0).

Deprecation notes are new to version 2.1 of the UCF, and we’ve done as good a job as possible back-filling them to ensure that we have covered our bases.

In a nutshell, when our mappers, reviewers, or approvers have made the decision to deprecate one of the records in the various XML tables, they will add their deprecation notes, their reasoning, to this field. There is no set format for what they are writing, so there aren’t any hard and fast editorial rules, other than something has to be added to the field during deprecation.

Date_Added is a date stamp for when the record was created.

This element is created when the record is entered into the UCF’s Master Content database and not the working database. We chose this method because the UCF team’s editorial process is a fluid one which allows, during the editing process, for records to be added, moved, deleted, or even “un-deleted” fluidly until the lock-date that ends the editorial process. Once the lock-date has been reached, all of the records are then finalized from the “working” list and uploaded as a batch to the Master Content database, which also triggers the change log process. Therefore, it is common to see all new records for any given quarter being added on the same date.

Because the Date Added element is controlled post-editorial process, the UCF database system manages everything automatically.

Date_Modified is a date stamp for when the record was modified. We use this as a key field for tracking all roll forward and roll backward field calculations. The initial date reflects the date the authority document was added to the database.

This element is created and updated when the record is entered into the UCF’s Master Content database and not the working database. We chose this method because the UCF team’s editorial process is a fluid one which allows, during the editing process, for records to be added, moved, deleted, or even “un-deleted” fluidly until the lock-date that ends the editorial process. Once the lock-date has been reached, all of the records are then finalized from the “working” list and uploaded as a batch to the Master Content database, which also triggers the change log process, which relies on this field to trigger that a change has taken place in the record. Therefore, it is common to see all new records for any given quarter being “modified” on the same date, and all modifications for the quarter to happen on the same date as well.

We have heard from multiple XML licensees that they would rather have the exact date and time that the record was modified instead of the batch upload date. That isn’t possible, given that all of the XML licensees also want us to produce a compact and digestible change log. A change log based upon the exact date of modification would have already produced several instances with over ten changes for certain records. Changes that were of no consequence to either the XML licensee or an end user, because those changes were simply a part of our internal editorial process. Therefore, to save processing time on the change log and to shorten the length (of the already very heavy) change log, we made the strategic decision to limit both date modified and date created to be the batch upload dates.

Because the Date Added element is controlled post-editorial process, the UCF database system manages everything automatically.

If the record is in a specific language, that’s what needs to be entered here. However, we are not using the name of the language, but rather the ISO 639-2 Codes for the Representation of Names of Languages reference. A complete and up-to-date reference can be found online at http://www.loc.gov/standards/iso639-2/php/code_changes.php. By default, all records are in English (code eng).

Because some of the records within the UCF are being provided by external sources, we now indicate this with a URI stored here. By default, the URI will point to Unified Compliance usage license information.

If the record is subject to external (outside of the UCF) usage terms, the URI will point you to those usage terms.

The Sort Value is relative to its siblings, sort ID is relative to the entire hierarchy. Developers should be using the Sort ID instead of the Sort Value.

Within the UCF, a record’s genealogy is a set of UCF IDs strung together as distinct words (e.g., 0000000 0000001 0000002) that represent (from right to left) the current record’s parent, grand-parent, great-grand-parent, on back to the very root element that spawned the list. At minimum, every record will have a genealogy of 0000000 which represents the root record within the list.

The genealogy element is initially created by the UCF database system when the record in question is created. If the record in question is moved lower or higher in the taxonomy, the genealogy is automatically re-calculated and the value will change to reflect the new taxonomic structure. Because the UCF editorial team does not have edit privileges for this element, the genealogy will always reflect the taxonomic position the record was last stored in. If there is a dispute about the record’s genealogy, the dispute is an editorial one, and not a programming one.

We sort our displayed information according to a taxonomic display hierarchy (which means that the genealogy plays a vital role). For the most port, each element in any of our lists is given a three digit sort identifier. We then append the record’s sort identifier to its parent’s sort identifier to create its Sort ID. We treat this numeric Sort ID as a text field so that we can run our sort routine from left to right in the character string.

There are some exceptions to the numeric Sort ID field, namely in the glossary and vendor lists wherein the Sort ID is actually the genealogical name of the record’s predecessors through its title. For instance, in the vendor list one of the vendors might be Sybari, which is a subsidiary of Microsoft. Therefore, its Sort ID would be “Microsoft Sybari”.

The Sort ID is created and managed in the same manner as the genealogy (it is a dynamic calculation). It directly reflects the record’s place within the taxonomic hierarchy and is therefore uneditable by the UCF’s editorial team (although the team does set the sort order, the system handles the ID to manage the sort order). Any disputes with the validity of the sort ID are in effect a dispute with where the UCF’s editorial team placed the record in question within the taxonomic structure.

Tthe colloquial, or database field name representation of the authority document’s official name. Because some authority documents will have names that are almost a hundred characters long. This is too long for representation in a vertically aligned spreadsheet cell and too complex because the official name also sometimes use reserved characters or words such as and, or, etc.

The name should be short, succinct, relevant to the authority document’s published name, and must ahere to database field naming conventions. Such conventions do not allow the use of certain characters. The list of restricted, nonusable characters in an authority document’s common name are as follows:

, + - * / ^ & = ≠ ( ) [ ] \ ; : $ AND OR NOT XOR TRUE FALSE

In addition, the first character in any authority document’s common name cannot be any of the following: space, period, or number.

The title of the Authority Document as given in

If there is any version information for the Authority Document, we list it here. This includes:

• Published date

• Released date

• Edition Date

• Effective Date

• Version identifier

As well as splitting out the basic information from the meta information, we have also created a complex type to define the official name of authority documents, which is a combination of the published name and the document's version.

The choices of elements within UCF_AD_Type are listed in their legal hierarchical status, as defined within the UCF_AD_Type_Type, documented below.

• Bill or Act

• Regulation or Statute

• Contractual Obligation

• Self-Regulatory Body Requirement

• Audit Guideline

• Safe Harbor

• International or National Standard

• Best Practice Guideline

• Organizational Directive

• Vendor Documentation

• Not Set

The Unique Resource Identifier (URI) of the authority document in question. This should be a direct link to the document (.pdf, .doc word format). If the document is secured, the URL should point to the document in such a way to redirect the end user through the sign-in page directly to the resource. If the document is a for-purchase document, the URL should point the end user directly to the purchase page for that specific resource.

This is a CSS formatted HTML entry containing the Authority Document’s Official Name, it’s ID, availability, the current number of citations associated with it, and when it was last reviewed and released.

The Title type tells the OEM programmer how to treat the entry. Remember the UCF’s Authority Document list is a hierarchical list. There are three title types:

CA = Category title only. This record only marks the authority documents that fall into its genealogy are a part of that particular category.

OR = This is the originating source of the Authority Documents that fall into its genealogy. For more information about originators and issuers, see the detailed info that follows in the meta data section.

AD = This is an Authority Document record, the meat, if you will, of the AD list.

GL = This is a Glossary Record, and will not have any controls assigned to it.

Some authority documents are freely available, some are restricted, some are only available through direct purchase, and some are available only to members of a group. This field should list the availability status of the authority document. The enumerations for the UCF_AD_Availability_Type are as follows:

• Free

• For Purchase

• Membership

• With Product

• For Purchase or With Membership

This is the category and originator of the Authority Document's parent. It is used for displaying the Authority Document in hierarchical order.

Within the UCF’s tracking of authority documents we have the problem of trying to figure out where a document came from. One would think that if the IRS wrote a procedure that they wanted you to follow, you could go to the IRS’s website, type in the procedure name, and have that document pop up on your screen. Fat chance of that happening. Funny enough, you can find IRS revenue procedures, but not at the IRS website – at 20 other websites. None of these folks are the publishers or the authors. So what do we list? Do we try to tell you the author/publisher of the IRS procedures is Wacky Tom’s Revenue Reporting site? We have to list something, so what we’ve done is split it into three elements – the originator (the IRS), the issuer (Wacky Tom’s), and the issuer’s URL (in case you don’t know where on the web Wacky Tom lives).

This field represents the status the document is in within the UCF’s document mapping process. The status fields are (in information flow order):

• Suggested meaning the document is at an early stage in the process. Much of the needed information is still missing but UCF staff are reviewing the document to decide if it should become “recommended” (see below).

• Recommended meaning the document has been recommended for inclusion in the UCF, but not yet verified.

• Queued meaning the document has been queued to be added to the UCF soon.

• In Research meaning the document is somewhere between being recommended and us figuring out what to do with it (or even find it in English).

• In Edit meaning the document is being worked on.

• Released meaning the document has been added to the UCF and should have an applicable release date.

• Not Applicable meaning the document has been verified to be a real authority document, however, the UCF mapping team have made the decision not to include the document in the UCF (for any number of reasons).

• On Hold Every so often we are sent an authority document for mapping that, during the mapping or research process, goes into change. Therefore, when that happens, we mark the authority document as being "On Hold" until we hear otherwise.

• Redacted means the record is no longer live. If the record belongs to an authority document (versus either a category record or originator record), then all the control citations within the UCF’s main tables will also be redacted.

This is the day, month, and year the authority document will take or took effect. There are differences between the various Authority Document types and where we found and how we gathered the effective date for each. For instance, because Bills or Acts are not yet codified into law, we listed their effective dates as the published date (or updated date) of the Bill or Act. Most regulations and statutes do list their effective date, and therefore our effective date is derived from their effective date. Contractual Obligations (like the various PCI documents), don’t list an effective date. Instead, they list a released date, and that’s what we used. For the rest of the Authority Documents, if we could find something that either stated “effective date” or “released date”, we used that. If not, we used the publication’s published date. For vendor documentation and best practice guidelines that are only published in HTML format, if the pages we mapped didn’t have anything listed as a date, we ignored their effective date. We’ve begun to run into documents that vary the effective date by section. For those documents, we will be using “Varies”. If we’re unable to determine an effective date, we will use “Not Set”. Only documents with status “Released” are required to have an effective date.

This is the day, month, and year the authority document was initially released or rereleased into the UCF’s database. If the UCF_Status field is anything other than Released, this field will have a null value.

This denotes the classes of Common Controls Hub accounts can see and utilize the Authority Document.

The price a Common Controls Hub account must pay for any "pay per use" Authority Documents.

This is the format that most, if not all, that the Citation Reference fields will follow. This is the kind of formatting, including any special characters or symbols, used to structure citations in the document.

It is only used in mapping projects.

Internal programming use only.

This is the record identifier of the Authority Document record that the Authority Document will supercede when published. It is set during the mapping process, and used by the UCF Mapper publishing engine.

The user must be able to read the document (at least at the skimming level) and determine of the contents of the document fit one of the subject matter areas covered by the UCF. The Common Controls Hub user must be able to search the CCH's catalog for Authority Documents by subject matter. This, therefore, is a list of all known subject matters that apply to Authority Documents in the UCF's library.

The list is dynamic. The subject matter list can be found HERE.

Internal programming use only.

The unique and persistent identifier for each record.

We use the id as the identifier so that if there is a discrepancy in how we any of the record’s information, any linked references to the record will not change. And as obvious from the previous sentence, we use the id field as the linking field when referencing this list from other lists.

The ID element is created when the record in question is created and is always assigned the next highest non-used, non-reserved ID in the system for that particular list.

We humans have to use numbers. However, when entering numbers, we humans also have a tendency to screw up the entry or copying of those numbers. A Dutch mathematician named Jacobus Verhoeff conducted a study of 12,000 numerical errors J. Verhoeff, Error Detecting Decimal Codes, Mathematical Centre Tract 29, The Mathematical Centre, Amsterdam, 1969, cited in Wagner and Putter, "Error Detecting Decimal Digits", CACM, Vol 32, No. 1 (January 1989), pp. 106-110. and from that, proposed a check digit calculation scheme http://www.augustana.ab.ca/~mohrj/algorithms/checkdigit.html#verhoeff that catches all single errors as well as all adjacent transpositions and most other errors.

To ensure that the IDs assigned by the system have integrity during input as well as distribution while being transferred into various formats (such as Excel, Word, Text, XML), each ID will also have its own checksum value stored in a checksum field.

Currently, the methodology for creating and verifying the checksum follows the Verhoeff calculation format.

The CheckDigit is created along with the record's ID as a calculation by the UCF database system. As such, once assigned it should never change because the ID will never change. A sample calculation format is shown in the use case scenarios.

The date and time the record was created.

The date and time the record was last updated.

An array of all of the Citation IDs associated with the Authority Document.

An array of all of the common names associated with the Authority Document.

This is the seven-digit ID that refers to the Issuer of the Authority Document.

Each record’s Parent ID, if the record can be listed in a hierarchical list. Knowing a record’s parent allows the OEM programmer to build multiple iterations of hierarchies.

This is the Authority Document's term in the Compliance Dictionary.

Internal programming use only.