The ability to convert data from one format to another is a crucial component of data transformation. All organisations have the need to use and share data across various systems and platforms. However, the diversity of data formats, protocols and sources can make the task of data conversion and distribution highly complex. FME Transformers can help overcome this challenge.
A key component of FME; the enterprise data integration platform, FME Transformers play a vital role in data transformation by enabling the manipulation and conversion of data from one format to another by applying various operations such as filtering, merging, splitting, and reprojecting data.
Highly customisable, FME Transformers help reduce manual effort, handle complex data conversion tasks and improve the overall quality and consistency of an organisation’s data.
With hundreds of FME Transformers available, they are highly customisable, help reduce manual effort, handle complex data conversion tasks with ease and improve overall data quality and consistency.
In this article we are going to explore the 10 most popular FME transformers as ranked by FME users* along with some favourites that our Locus team like to work with every day.
The Top 10 FME Transformers – Ranked by Popular Usage
- AttributeManager. AttributeManager can manipulate and manage attribute-related tasks within your FME workflows including adding, renaming, deleting, calculating and updating values based on specific conditions. Analysts working in the transportation industry may use the AttributeManager to convert a departure time attribute to a more user-friendly format (i.e. “2023-04-03T08:30:00-5:00” becomes “8:30AM”and calculate the arrival time of each trip by adding the duration of the trip to the departure time
- Tester. The Tester Transformer is used to test data, routing features in your workspace according to test condition(s). The tester is great at helping check data quality, dividing features into two streams and isolating features of interest. If used early in the development process this transformer can save you a lot of time further down the line!
- AttributeCreator. Create new attributes in your data with the AttributeCreator. Attributes are fields or columns in your data that store information about the features or objects represented. One of the primary functions of AttributeCreator is to create new attributes based on existing attributes. You can also specify the data type, length, and precision of the new attributes, ensuring that they are compatible with your existing data
- FeatureMerger. Use FME Transformer FeatureMerger to combine data from two or more sources based on spatial and non-spatial relationships. Functions can include a spatial join or an attribute join for example data from a CSV file can be joined with data from a database table based on a shared attribute such as a customer ID. Using FeatureMerger for example, a local government department can track the status of building permits and consents in their area by extracting data from the consent system and loading it into a dashboard having combined the data with other useful data sets on property information and zoning
- AttributeKeeper. Maintain and preserve specific attributes throughout your workspace. With this transformer you can choose which attributes to keep and FME will do just that, ensuring those attributes are passed through each transformation in your workflow
- TestFilter. Set-up multiple filters based on criteria, such as attribute values and run your tests with only the data that meets that criteria. A recently added parameter in the TestFilter allows you to control which ports features will make-up the output, following a passed test – first passing port (default) or, all passing ports. Another excellent tool for delivering functional workspaces
- Creator: Extend the functionality of FME and create your own custom solution by defining the inputs and outputs of your transformer as well as the logic applied to transform the data or simply trigger a transformer e.g. a HTTPCaller or FeatureReader
- DuplicateFilter. Detects duplicates based on criteria that you specify i.e. attribute values, and then remove any features that match that criteria. A terrific transformer for streamlining processing tasks. This transformer has universal industry applicability but let’s consider an example in the healthcare sector where duplicate datasets of patient medical records may exist. To remove the duplicates, the DuplicateFilter transformer can setup filters based on attributes like name and date of birth and remove any duplicate features that match that criteria.
- FeatureJoiner. Join multiple input feature streams into a single output stream based on a common attribute to deliver a comprehensive dataset
- Aggregator. This transformer can be used to calculate sums, averages, minimums, maximums, and counts, grouping data based on specific criteria. The Aggregator transformer can group data at multiple levels as well as spatially based on data relationships
It’s amazing what you can do with the FME Transformer SystemCaller! I can run any application via windows command and automate tasks within FME – the capability of this transformer makes it one of my favourites. Darren Fergus | FME Data Solutions, Locus
Bonus Transformers that the Locus Team Love
- FeatureReader and FeatureWriter. Used to read from and write to databases. FME Transformer FeatureReader can read data from many formats that FME supports and Feature Writer can write it to any other format. These transformers are highly configurable and allow you to filter data based on specific criteria like using a spatial relationship or a where clause
- SpatialFilter. Filter features based on their location, selecting features within a certain distance of another or clip features based on their spatial relationship. For example, you may have a large dataset of polygons representing different geographical regions but you only want to focus on one area. Use the SpatialFilter to clip the dataset based on a boundary
Seems like an obvious choice but the AttributeManager is my favourite. It’s the do-er of all things – you can carry out the function of many individual transformers in one single transformer – so versatile and helps keep a workspace neat and tidy. Pip Norris | FME Data Solutions, Locus
- HttpCaller. The HttpCaller automates the retrieval and transformation of data from web services, web pages and APIs enabling it to be easily integrated with other data sources. Working with FME In the utility industry, an organisation could use HttpCaller to retrieve weather data from an API and integrate it with meter data in order to better understand energy consumption pattern
- SystemCaller. Execute system commands, scripts and programs from within FME workspaces providing the ability to integrate a wide range of external functionality into your FME workflow
There are so many powerful FME transformers available that it’s hard to pick just one, but if I had to… it’d be the HttpCaller – it allows us to connect to just about any REST API (and lots of non-REST ones, too) so we can pull in thousands of data sets covering all sorts of topics. Jan Roggisch | FME Data Solutions, Locus
Whatever your data transformation need, FME can accomplish it with ease. A comprehensive toolset, FME transformers can manipulate and restructure data across various formats, sources and protocols to deliver a single, unified dataset – quickly and reliably. Try FME for free now and discover how to get the most value out of your data with the help of FME Transformers.
*FME transformer usage rankings as provided by Safe Software in their Safe.com FME Transformer Gallery
