Overview
Data Refresh is a simple, automated utility, created to alleviate the tedious problem of manually synchronising database contents between environments.
Once installed, an export profile is required to be configured, comprising table selections and obfuscation configurations. After nominating the tables to be included in the profile, an array of obfuscation methods are made available, including fixed-length randomisation, preserved-length (respective to the column definition) randomisation, or substitution with a static value. These methods are implemented at a column-level, ensuring sensitive production data can remain confidential, whilst still migrating realistic, manageable values.
Initiating a profile will return a progress gauge, depicting the overall process completion in real-time. Meanwhile, the requested database content is being fetched, assembled, obfuscated, consolidated and zipped. These stages can be relatively resource intensive, however we’ve sufficiently balanced the load through both batch-processing (with each new process being instantiated on a new Appian engine) and file rolling (throttling based on line/file-size limits).
Upon completion, the target system will attempt to import the prepared package, though not without the requisite password being provided, safeguarding against accidental or unintended imports. Additionally, as a further security measure, passwords are algorithmically hashed and time-sensitive. Data Refresh also allows one-click, flat-file backups of all SQL (DML/DDL) associated with a given profile, which (much like imports and exports) is password protected.
All functionality listed above pertains to the launch version of Data Refresh, with plans for new obfuscation types, non-MySQL database support and additional features in future releases.
Key Features & Functionality