FME Workspace Statistics (You Can Help!)

markat亚搏在线safe — Tue,20 Nov 2018 20:08:47 GMT

Introduction to FME Workspace Statistics

Workspace statistics help give you an idea of how you,and others in your organization,are using FME.This will allow you to:

focus organizational training on commonly used functionality (formats or transformers).
followbest practiceson how to build and structure workspaces in a uniform way within your organization.
help document your workspaces.
identify workspaces for possible improvement,including transformers that can be updated.
make a more comprehensive upgrade plan by understanding which transformers can be updated.
help 亚搏在线Safe understand your FME usage so we can improve the FME 亚搏在线娱乐平台products appropriately.

Attached is a workspace that will read one or more workspaces and generate an Excel spreadsheet with a variety of workspace statistics including:

summary statistics for each reader/writer format and transformers (includes readers/writers in transformers)
summary statistics for each workspace (number of readers/writer/transformers etc.)
list thetransformer versionsand compare them to the transformer version of the current version of FME
detailed statistics for each workspace (optional)
connectivity between transformers with frequency counts

Output is an Excel spreadsheet with the following sheets:

Header: General information about this analysis.
Workspace Summary: summary statistics for each workspace,including last save build number.Workspace names can be obfuscated.
Reader/Writer Summaries: These give the format counts for all the readers & writers for all workspaces,including readers and writers in transformers such as FeatureReader.There is a second pair of sheets that have the statistics for families of formats (i.e.Oracle includes Oracle spatial,Oracle non-spatial etc.).
Transformer Summary: Counts for each transformer (and optionally custom transformers).Includes the list of transformer versions encountered in the workspaces and the current transformer version - this can help you decide if it's worth updating some transformers.Embedded custom transformers will have their contents included in the analysis.Linked Custom Transformers will not have their contents included,unless you include the *.fmx files.
ConnectionSummary: this shows a summary of the from-to connections between transformers (or readers/writers) and their frequency.
Readers/Writers/Transformers by Workspace(optional) - this shows the breakdown of functionality by workspace (workspace names an be obfuscated).

Downloads

File Copy Workspace:FileCopyWorkspace.fmw(run in 2018.0 or higher)

Statistics Workspace:WorkspaceStatistics.fmw(run in 2018.0 or higher)

Example results:WorkspaceStatistics_Webinars.zip

If you make any changes to the workspaces,or find any problems,please share your changes and thoughts here so everyone can benefit .

Share your results

Please share your results with us atfmestats@亚搏在线safe.com- this will help us determine which formats and transformers are more important to FME users.Please differentiate between a desktop analysis and an FME Server repository analysis.

The statistics workspace allows you to obfuscate the workspace names (optional) if you have sensitive projects (recommended).You can also choose to ignore Custom Transformers (if these are sensitive).We will consolidate results from all users and publish them here.

How to use

Run the Statistics Workspace (WorkspaceStatistics.fmw): Point at a directory contains your workspaces (or at a root directory with a set of directories containing workspaces).The Statistics Workspace will then get the statistics for all the workspaces and/or custom transformers it finds.

It's more efficient to point to a single directory,rather than scanning all directories on your system (this can be pretty slow).To help with this we've included a second workspace (FileCopyWorkspace.fmw) that will create a catalog of your workspaces by copying all .fmw/.fmx files from a set of folders into a single directory.The Workspace Reader (FMW) may fail if an invalid or corrupt workspace is read - which is why it's better to copy all the workspaces into a single folder as you can then remove or investigate the problem workspace.

File Copy

Run the FileCopyWorkspace.fmw in two steps.

Enable the PATH reader and use the File Browser to point at your ‘root' directory.The File and DirectoryPath reader will scan all directories below this root folder and create a catalog of your FME workspaces.Edit the catalog to remove any duplicate workspaces,etc.
Disable the PATH reader and enable the Excel reader.The workspace will read the workspace catalog and copy all the workspaces into a common folder.

Statistics Workspace

Input prompts:

Your Name,组织,Comments (all optional)
Hide workspace Names: Select Yes if you would prefer workspace names to be hidden
Is this an FME Server Repository.Yes|No This will help us understand FME Server usage a little.
Include Disabled objects: Yes | No
Include workspace Detailed Statistics: Yes | No.Set to ‘No' if you'd prefer not to share details of individual workspaces.
Include Custom Transformers: Yes|No
Source FME Workspace Directory.Use the Select Multiple Folders/Files Browserif you skipped the file copy and intend to scan multiple folders.
Destination Microsoft Excel File.Excel report location.

Usage Notes

Try and avoid creating statistics of versions of workspaces,i.e.workspaceV01.fmw,workspaceV02.fmw
If you use FME 2019 then you can also read FME Template (*.fmwt) files.If you're using FME 2018 you'll have to unpack FME Template files.Use Winzip wzunzip (or a similar application) to do this.You can do it on the command line: wzunzip *.fmwtYou'll get all the contents (data + .fmw) but you'll be able to clean-up
Please indicate if these statistics are for an FME Server repository (otherwise we'll assume they are desktop statistics).Generally in the FME Server workspaces are in the repository folder,which is the file share,not the install folder,i.e.C:\ProgramData\亚搏在线Safe Software\FME Server\repositories\ where C:\ProgramData\
If you use FME 2019 then you can also read FME Template (*.fmwt) files

Results

Reader / writer counts.If there are multiple readers / writers to the same workspace - this is counted as one reader or writer in the reader/writer summary stats.
Summary of connections: connections between all reader feature type to all writer feature type is counted as a single connection - reader to writer.
Summary connections skip junctions and ‘tunnels'.

FAQ

What's the difference to usage stats?
When you install FME,you can opt to send us usage statistics.This gives us a picture of what formats and transformers you're adding to your workspace.But some workspaces rarely get edited,they just run - particularly those workspaces you might have on a production server.We think these are probably the most important workspaces you have and we're not getting their statistics.
Transformer Connections
There are more transformer connections available on theTransformer Gallery.

Analysis

We have run the workspace statistics for example workspaces used in 亚搏在线Safe webinars to illustrate the results - attached.

Attachments