For enterprise GDA2020 migrations, the volume of data is too great to undertake the process manually. As a result, for the migration process we recommend the use of Python through all stages to automate the process. There are other tools that can handle the data transformation in an automated fashion, such as FME.  However, Python can be universally used across the entire process – from data auditing through to automated republishing of services.

At a high-level, the data migration process involves 3 main steps.

1. Content audit
2. Planning strategy for data migration
3. Data transformation and check

Let’s discuss them in detail.

A content audit is the most crucial part in the migration process as it uncovers all sources of data, data relationships, and inconsistencies that impact the final output. Additionally, it should uncover your data properties such as editor tracking, versioning, replicas, data owners etc.

While data is vital to this process, the content audit is about so much more than just the data. It is crucial to understand the map services your data is present in (and the related MXD/APRX), the portal web maps and applications that reference these map services, integrations with other systems etc. This will help us at the end of the transformation process to verify that it has been successfully completed and to ensure that all data is updated to the new datum.

The content auditing process involves identifying all the data types, sources, and formats involved in the migration process and determining their datum, accuracy and quality. The preparation of a comprehensive data transition plan is also essential, which outlines timelines, budgets, resources, as well as risk management and communication strategies.

Crucial information to uncover;

When planning for the transition, decision-makers must choose between transforming datasets on the same server and database or setting up a new server and database to create new datasets in GDA2020. Both strategies have their benefits and limitations, and organisations must evaluate their unique circumstances before deciding which approach to take. A crucial factor in making this decision is the length of the change freeze period required for the transformation process.

The first strategy involves transforming datasets on the same server or database, which is only suitable for a small number of datasets. This approach is advantageous in that it is relatively easy to carry out and requires minimal disruption to operations. However, it may not be practical for large datasets or in situations where multiple datasets need to be transformed, as it could take a considerable amount of time to complete. This creates a significant interruption to service delivery and could result in unacceptable downtime for critical services.

The second strategy, preferred by many organisations, is setting up a new server or database to create new datasets in GDA2020. This approach is suitable for larger volumes of data or datasets that require frequent updates. By transforming the data in advance, it is easier to minimize the impact on service delivery and the transformation can be tested and checked in advance. Another advantage of this strategy is that it allows the organisation to retain control over the production environment, thus ensuring data security and integrity. The main disadvantage of this strategy is that it requires a more considerable investment in resources, including time and effort.

ESRI products (ArcMap 10.6.1 or later and ArcGIS Pro) provide a range of transformation methods (7 parameter transformation, NTv2 Conformal transformation, NTv2 Conformal + Distortion transformation) to migrate data from GDA94 to GDA2020. Depending upon type of the data i.e. local, statewide or nationwide data and accuracy requirement one has to select appropriate transformation method to migrate data into GDA2020 datum.

It is imperative to test the data transformation process to ensure the transformation works as expected. It also provides an opportunity to evaluate the migration plan and refine it according to the findings. This is a critical step as every spatial environment has its own nuances that need to be accounted for. Once the transformation process is confirmed, a change freeze or outage is required in the production environment to carry out the transformation. This ensures that no changes occur to data between when it is transformed and when it is ready to go live.

Once your datasets are transformed, you can apply other functionality like editor tracking, versioning, relationships, and more based on the auditing of the datasets. Finally, don’t forget to update your ArcGIS files (.aprx or .mxd) and re-publish any relevant services. This will ensure that your data is accessible and up-to-date no matter where you’re working from.