An organization works withmany systems that each hold their own data, often in different data formats. When data comes from multiple sources, data integration is an integral part of a team’s processes.Often the data is siloed in systems, existing independent of one another within a business. Bringing that data together means it can be used in meaningful ways and to help inform business decisions. Integration and automation also give your team time to focus on tasks that are beyond the everyday functions of the business, validating the quality of the data and delivering an up-to-date single source of truth, saving you time and resources.
Applying FME to Utilities
Auckland Council is responsible for managing thousands of assets across New Zealand’s most populated city. From Utility companies often have large amounts of data from multiple sources that need to be converted, transformed and validated. FME allows these companies to accomplish a variety of automation tasks across multiple workstreams, some of these include:
- Design and Planning
- Company Operations
- Asset Management
- Service Outages and Leaks
- Distribution Networks
- Cross Team Communication and Collaboration
- Information Sharing Between the Public, Shareholders and Others
1. Data Migration
Utility companies often need to migrate incoming data or data from a legacy system, converting it and integrating it with new data, as a repeatable task.
- Task: Data migration between systems where integration from GIS to CAD or CAD to GIS needs to be carried out on a regular basis, as a repeatable migration workflow and which can be easily automated using FME
- Solution: Building a migration workflow that connects to your systems
- Result: Data is written out to where it needs to be
2. Data Consolidation and Quality Control
Utility companies regularly collect and analyze data from different sources whether it be from out in the field or from multiple departments within the organisation to meet stakeholder requirements. Running an automated self-serve validation portal process allows your organisation to attain accurate, complete and compliant data
- Task: Data validation and quality control
- Problem: Data needs to be good quality and of value to an organisation
- Solution: Implement data portal self-serve for data validation purposes. Workspaces can use FME’s validation or data quality transformers – which includes the geometry validator or attribute validator. Giving the ability to then generate reports, PDFs, Spreadsheets or HTML pages that can be automatically delivered via email
- Result: Data is now correct, complete and compliant to industry or organisation standards and can be used throughout the business
3. Synchronising Data and Integration of Data
Decision makers across businesses require a complete picture of all available information to reach conclusions and make recommendations. This can be achieved through the integration of multiple systems and the delivery of automated workflows. Data integration is carried out between systems and applications from different parts of a business, external and internal sources or geospatial and business software. The output of these systems including; GIS, CAD, Bentley and Microsoft are brought together to a central repository (or kept separate and synchronized) while needing to be kept up to date and coordinated.
- Task: Synchronize data across different systems and formats e.g. GIS, CAD, Bentley, Microsoft
- Problem: Difficult to make decisions if you don’t have all the data available to you
- Solution: Use FME to build data and application integration workflows
- Result: Synchronized and up to date data from across systems that is used to make informed decisions
A workflow that builds integration between data and applications can run as an automation on FME server to synchronize the data automatically.
4. Information Delivery and Distribution
Utility companies often have self-service portals for stakeholders to access data. This provides high quality data in the format required by the general public, customers, design and contractors or other clients and is used to inform the public about scheduled or unscheduled outages or in times when emergency management is needed.
- Task: Data needs to be accessible and distributed to shareholders and the public
- Problem: Critical information demands to be delivered and distributed in a range of formats suitable to the end user
- Solution: Data integration workflow to collect data into appropriate formats and hosted on an FME Flow Hosted portal for sharing
- Result: User friendly data formats that are accessible at the self-service portal
Generate Report Workflow: Water Utility Company
The FME platform is used to generate reports for technicians, where information is captured about network issues. The workspace pulls together detail about the leaks and locations of the water mains. The extent and severity are also captured while information around the nearest located valve is included so they can be turned off.
A simple workflow like this can run as an automation on FME server. The automation would be set up to listen for events or triggers that initiate the workspace. This can be done in a host of ways i.e. scheduled to run at a particular time, activate when a file is added to aDropbox account, via a Webhook or a directory and sending the output to an email.
5. Beyond Basic Data Types
FME allows you to easily experiment with integrating data and delivering the output to different formats/functions or run analytics. Some of the exciting data sources that could be leveraged include:
- Drones/UAV
- Augmented Reality
- Satellite Imagery
- Game Engines
- Cognitive Services and Machine Learning
- Virtual Reality and Interactive Web Visualisations
- Sensor Data
Many businesses have similar workflows that are often repeated. FME gives you the power to automate simple tasks and perform more complex data integration modelling that improve operational efficiency and enable data-based decision making.
There are FME licensing and subscription options to suit businesses and budgets of all sizes. Chat to Locus to get started.
This article is a synopsis of the Safe Software Webinar; How Utilities Can Improve Operations Using Data Integration Workflows (recorded June 2020), reviewed and repurposed by Pip Norris, FME Data Solutions, Locus
