Extract, Transform and Load
Make your data management easier.
Why you need it
Too many different sources and types of data? Businesses often have data in a huge variety of different formats. The data is consumed by many business systems, and those systems are usually capable of reading data in specific file formats only. Manipulating and converting data for all those internal databases can be a difficult and time-consuming task. Add in spatial data and the need for Quality Assurance in the mix and things can get even more complicated.
ETL (Extract/Transform/Load) allows you to manage all or parts of your data manipulation much more efficiently. This is achieved by using advanced ETL functions. ETL makes the dream of integrating all your data into a “single point of truth” database a reality.
How does it work?
The three main steps are what gives the process its name:
Extract: Take data from a variety of sources/databases.
Transform: Translate data from one file type into another, while ensuring quality is maintained.
Load: Add the data to the target database.
This widely used process leads to effective management of information and workflows, giving you consistent and accurate data processing and storage.
How can GIS People help?
The team at GIS People includes industry experts in the design and implementation of ETL processes. We offer a comprehensive ETL consultancy service to help you maximise data workflows and quality.
There’s no need to worry about all the technical aspects – we can handle that.
Want some application examples? Read on.
Example 1: Near real-time data feeds
We can design a system that quickly delivers up-to-date spatial and related data. E.g. for transport networks, emergency flood/fire management or disease outbreaks.
This data often comes from near real-time data feeds or from the field. GIS People will extract, transform and load data into a format that can be read and analysed by GIS software. We have particular expertise in indexing and optimising large volumes of data, and geocoding for fast address lookups. This gives users better and quicker understanding of the data, helping rapid decision-making and management.
Example 2: System migration
Do you need to migrate your geospatial data to a new GIS or integrated system?
Transferring data from one software package to another can be a hugely time-consuming process. There are often hundreds of layers in different formats, and legacy issues to deal with. We use ETL to clean and standardise the data, making the whole migration process efficient with Quality Control to guarantee high data accuracy.
We also offer Basic and Advanced training in the ETL tool “FME”, delivered by a Safe Software FME-Accredited trainer. This gives your staff the skills and power to perform their own FME ETL workflows – see our dedicated training website GIS College for more information.