Data Exchange

 

Process Overview
When the export process is initiated, it scans for any changes in subscription data and audit logs, writing the updated information to a data server in JSON format. On the receiving end, each system’s import process checks the data server for relevant updates that need to be applied to its database. Users are then able to review and approve the data changes queued for import. The import process intelligently handles differences between the local system’s data structure and the incoming data sent by the publisher.

​

Key Features

While there are many detailed features and controls involved in automated data exchange, the key high-level considerations include:

• Structural Flexibility: Data exchange accommodates scenarios where either the governor or the subscriber has altered the table structures or associated rules. For example, if the governing system adds a new column to a table, subscriber systems can continue to receive updates without interruption, even if they haven't yet updated their structure to include the new column.

• Support for Temporal and Non-Temporal Data: The mechanism supports subscriptions for both time-sensitive (temporal) and static (non-temporal) data, ensuring data consistency across varied scenarios.

• Column Redaction Options: Redacting sensitive information is straightforward with options to automatically mask, substitute, or null out specific columns as needed.

• Triggering of Data Exchange: The platform triggers data exchange whenever a record or a related group of records is updated. For instance, a record group for a contract could include a contract header along with associated terms and participant details.

• Data Preview During Import: When importing data, users can preview changes, seeing before-and-after snapshots of all modified values, which helps them assess the impact of updates before approval.

• Visual Indicators for Pending Updates: When users browse data in their system, the platform provides front-end cues highlighting records with pending updates from the governing system.

• Subscriber Group Management: The platform simplifies subscription management by organizing subscribers into groups, reducing complexity. For instance, a new regional outlet added to a regional subscriber group automatically inherits all relevant subscriptions without additional setup.

• Master Data Exchange Focus: The primary focus is on master data exchange. For transactional data, a different process is used. For example, service providers might exchange transactional updates with customers through a custom interface.

• Data Integrity Management: The import process detects data integrity violations (e.g., foreign key, null constraints) and allows users to either resolve these issues before processing or ignore the update and store comments explaining the exception.

• Automation vs. Manual Approval: The import process can be fully automated or require manual review, depending on predefined rules based on the source system and the type of data being imported.

• Workflow Integration: The data exchange mechanism can interact with a workflow engine, triggering related processes. For example, a government department subscribing to citizen registry data could automatically initiate workflows when a citizen changes address.

 

Inheritance

The platform also supports hierarchical inheritance. For example, a GL account structure could be defined by an industry body (e.g., CPA) and then inherited by industry-specific models, such as oil and gas, which could further extend and customize it. This hierarchical setup can extend down multiple levels, with each inheriting and adapting shared data as needed.

 

Subnets for Enhanced Data Sharing

Each system operates within a subnet server. Small organizations typically run a single system on their subnet, while conglomerates may operate multiple systems. Data within a subnet is shared instantaneously among systems, enhancing performance and enabling secure data sharing within a conglomerate. Subnets can communicate with other subnets via a global master data server.

 

Stub systems for exchanging data with legacy systems

The platform uses stub systems to bridge gaps between conforming and legacy systems. Once an integration is developed with a legacy, it becomes reusable across multiple organizations, simplifying future integration projects and enhancing the platform’s interoperability.

 

Custom Data Exchange

While the built-in data exchange mechanism covers around 90% of integration needs, custom data exchanges can be programmed to address specialized requirements. Scenarios requiring high-performance data sharing or secure data exchange between systems are easily accommodated through custom integrations.

 

Accommodating Monolithic Systems

The platform also supports monolithic systems for conglomerates. Sometimes organizations have large enterprise systems with high volumes of data and transactions and are regionalized to further complicate the situation.

The 3D Platform allows these systems to be broken into smaller systems that are all continuously exchanging data. Each of the systems will have its own system ID and will generate data according to that system ID.

A manual process can be set up to instantaneously exchange data between the systems. It can even have flexibility to allow data to be modified to meet unique needs and can limit the data exchange to only tables and data that are of interest to be exchanged.

Furthermore, it will be possible to aggregate all data from these systems into a data warehouse for reporting.

 

Data Exchange Using QR Codes

Data exchange can also be achieved manually using QR codes (or equivalent technology). QR codes printed on documents contain the core data for a record, including three components: Dictionary Table ID, Row ID, and Header Data. This allows the unique identification of any record within any system, regardless of where it was created.

For example, a QR code on an oil well could be scanned by a service provider to instantly access well details. Any recorded service data would then be linked to the correct well when transmitted to the oil company, streamlining the entire process.

​

One-Time Secure Subscriptions

In critical situations, such as healthcare emergencies, one-time secure subscriptions allow for rapid access to essential data without copying detailed records to the subscriber’s system. For instance, a healthcare provider could authenticate through a patient’s device to access vital medical records, receiving both core patient data and a temporary graphical image of more detailed health information.

​

Benefits

• Simplified Integrations: Parameter-driven configurations replace custom coding, accelerating integration setup and reducing costs.

• Custom Integrations: While automated, the system is flexible enough to accommodate custom integration tools for specialized scenarios.

• Master Data Management: The platform streamlines master data management by ensuring consistent data across systems.

• Support for Transactional Changes: Updates are processed as grouped transactions, reducing the risk of incomplete changes.

• Controlled Import Processes: Imports can be tailored for full automation or manual review, depending on organizational needs.

• Error Reduction and Consistency: Automated data exchange ensures consistent data across systems, minimizing errors and inconsistencies.

• Improved Efficiency: Quick and efficient data transfers reduce manual entry and streamline operations.

• Support for Monolithic Systems: Allows monolithic systems to be broken into smaller systems that communicate.

• Enhanced Collaboration: Teams across systems can work together more effectively, sharing data seamlessly.

• Scalable Design: The platform scales to handle growing data volumes as organizations expand, future-proofing data management.

 

The 3D Platform’s data exchange mechanism is a transformative solution for organizations seeking seamless, secure, and scalable integration across diverse systems.