ABSTRACT
The Fact pipe shares much in common with the Definition pipe, particularly with respect to the requirement that in order to load the facts, they’ll need to be married to the Reference pipe for the dimensional connections. Where the data in the Definition pipe only need to be married to the one-dimensional Reference for the single Target dimension, the facts will ultimately need to be connected to all of the dimensions, only some of which will have been sourced into the Reference pipe. Specifically, the processing covered in the Fact pipe includes
◾ Transforming source-only intake data into source-to-target staged data through a join to the Metadata dimension, including Codeset translation of nonstandard source values
◾ Generating source-to-target staged data through a join to the Metadata dimension to pick up both valued and factless facts that have been defined against any inbound datasets
◾ Consolidating the roles, ranks, and weights of the reference entries for each fact in order to derive a single Group Composite for each sourced dimension for each fact
◾ Resolving a Group ID for each generated Group Composite, either because the Group ID already exists in the dimension or because it is created to support the current load
◾ Setting the Data State and Datafeed dimensions to the appropriately sourced values
◾ Resolving unsourced dimension entries for required dimensions, including resolving the Organization dimension to the sourced Enterprise ID, the Calendar dimension to the date in the Source Timestamp, and the Unit of Measure dimensions to Factless, Numeric, or Text depending upon an inspection of the actual fact value being processed
◾ Resolving all remaining unsourced dimensions to the Not Applicable entry in the dimension
◾ Inserting new fact values into the Fact table (recalling that handling multiple Fact value columns in multiple Fact tables has been deferred to the Gamma version)
◾ Changing the data state of older facts that might have been superseded by any newly arriving facts, including marking these new facts as Superseded if they are older than current instances of those facts from previous loads
The high-level sequence diagram in Figure 12.1 illustrates these main steps in the processing. As with the Reference and Definition pipes described in the prior two chapters, the Fact pipe begins with the metadata transformation processing.
