Introduction

Welcome to the FME Desktop for Esri's ArcGIS Pro Utility Network Migration Training Resource Center.FME Workspaces are available to help migrate your Esri ArcGIS Geodatabase to the new ArcGIS Pro Utility Network (UN) data model.This is a self-directed resource for understanding the FME Utility Network Migration workspaces.An overview of the ArcGIS Pro Utility Network can be foundhere.

This self-directed resource has three parts:

• Part 1: FME Workbench Basics.What you need to know about FME Workbench and Data Inspector to create and modify FME Workspaces used for Utility Network migration.
• Part 2: Key Transformers.Understand the key transformers used in the Utility Network migration workspaces.
• Part 3: Utility Network Migration FME Workspaces.Review of the Utility Network migration workspaces and how to modify them to meet your needs.

If interested,亚搏在线Safe Software will be offering a free live online training course:FME & Esri Utility Network Migration,December 3-6,2018.

Downloads

Course recordings are availablehere,corresponding to the parts of the course listed below.

Before you start

Prerequisites

• Understanding of the ArcGIS Pro Utility Networks
• Domain Specific Asset Package(s)
  • Water
  • Gas
  • Electric
• Utility Network Solutions Tools
• Source Data (Pre-Cleaned)
• FME Workbench available through either:
  • ArcGIS Data Interoperability Extension*
  • ArcGIS Pro 2.1+ (Water & Gas)
  • ArcGIS Pro 2.2+ (Electric)
  • ArcGIS Desktop 10.6
  • ArcGIS Enterprise 10.6
  • 亚搏在线Safe Software FME Desktop Esri Edition
    • FME 2017.1.1.0 Build 17650+ (Water & Gas) & ArcGIS
    • FME 2018.0.0.2 Build 18301+ (Electric) & ArcGIS

Downloads

• Domain Specific Utility Network Workbench Template(s)

Part 1 - FME Workbench Basics

Introduction to FME: In this part of the training,you'll learn the basic operations of FME Workbench (or in the Interoperability Extension) that you need to know so that you can edit,run and test the migration workspaces.If you are already an advanced user of FME you can skip Part 1.

See the tutorialFME Desktop Data Integration.

If you require additional resources to become more familiar with FME,please find additional training courseshere.

The migration workspaces also use the following functionality that you need to be familiar with:

• Updating Reader & Writer Feature Typesdocumentation
• Junctions and Hidden Connections

Part 2 - Key Transformers

In the following section,some of the key transformers that you will encounter throughout the Migration Workspaces are listed,along with a description of the Transformer's function.Please refer to our documentation foundhereor our Knowledge Centre foundhereif you require additional resources.

Attribute Handling

• AttributeManager/AttributeCreator
  • The Attribute Manager and AttributeCreator are used for renaming,removing and setting attribute values.The AttributeCreator is also used to set the values using if-then conditionals.
  • Advanced logic/conditionals are supported in the TestFilter,AttributeCreator and the AttributeManager.Working withConditionalsexample.

Schema Mapping

• Database Joiner
  • The DatabaseJoiner is used to join attributes from an external feature to a feature already in the workspace.The DatabaseJoiner is able to query an external table and retrieve attributes associated with that feature.
• FeatureMerger/FeatureJoiner
  • The FeatureMerger is used to merge two features that already exist in the workspace together.The transformer requires two inputs,a Requestor,and a Supplier.The Requestor is the feature that will receive the new attributes and/or geometry,while the Supplier will provide the attribute and/geometry to be merged.
• SchemaMapper
  • The SchemaMapper is used to convert the existing schema of a feature into a new structure through the mappings defined in an external lookup table.The lookups are defined by a series of conditions/filters,and a series of actions that will be performed when those conditions are met.This will allow for new Attribute creation,assignment,and renaming.
• AttributeValueMapper
  • The AttributeValueMapper is used to compare attribute values to a lookup table,and assigning a new value where a match is found.As such,this can be used to map a subtype or domain.The matched value can be written to a new attribute,or overwrite an existing attribute.This can be used as an alternative to the SchemaMapper,however,multiple AttributeValueMappers can often be replaced with a single SchemaMapper.

Geometry Handling

• PointOnLineOver layer
  • The PointOnLineOverlayer performs a spatial join between a point and the line it is connected to.Each line will be split at the closest place to any point(within the specified tolerance) - attributes can then be shared between the two features.Attributes can be shared between the Point and Lines through the creation of a List for the desired Output(Point attributes on the Line feature or Line attributes on the Point feature).
    • Understanding List Attributes documentation
    • Working with List Attributes
    • Creating Lists using Transformersexample 1
    • Creating Lists using Transformersexample 2
  • TheSpatialRelator,TopologyBuilder,and theNeighborFindertransformers are also used to create spatial joins in the migration workspaces
• AnchoredSnapper
  • The AnchoredSnapper will perform a snapping operation on features that match the input specification if they are within the specified tolerance.This is primarily used as a cleaning operation,or to ensure that features are correctly snapped to a line.This transformer is similar to the Snapper transformer,however,the AnchoredSnapper uses Anchor features that are considered accurate and thus will not move during the translation.
• Clipper
  • The Clipper is used to perform a geometric clipping operation.A Spatial Join can be performed to share attributes between the Clippee and the Clipper features.

Advanced Transformers

• List Transformers.
  • Severallist handling transformersare used in the migration workspaces: ListHistogrammer,ListSorter,ListElementCounter,and ListSearcher
• VariableSetter/ VariableRetriever
  • TheVariableSetteris used to set a Variable to the Specified Value,this variable can later be read back in using the VariableRetriever.
  • TheVariableRetrieverwill read the specified Variable (set from the VariableSetter) and place the value into the specified attribute.

Part 3 - UN Migration Workspaces

Links to the domain specific (gas,and water) Utility Network Migration Tools can be foundhere.This will direct you to the Esri Utility Network Migration Tools where you can download the resources you need for the migration from ArcGIS to the Utility Network.This includes:

• Documentation
• Domain-specific migration workspaces and related schema mapping spreadsheets
• Sample data

Refer to the PDF documentation for your domain-specific migration tools located within the Documentation directory of your downloaded Migration Tool package such as "Gas Utility Network Migration Tools Documentation" or "Water Utility Network Migration Tools Documentation".

Planning

There are a couple of important questions to ask yourself,before starting your migration.

• Is your source ArcGIS data ready to be migrated?
• 你打算如何执行这个迁移吗?Are you going to migrate circuit by circuit,or as a single Bulk translation?
• How will you validate the migrated data?
• Have you allocated the appropriate resources for migration and validation?

Plan a trial migration: a recommended approach is to take a single circuit,or pressure zone then validate,migrate,load,and assess the results.Determine if the current configuration matches your desired or expected results.

预迁移

• Download asset package.
  • Electric
  • Water
  • Gas
• Perform a data quality check on your existing ArcGIS Geodatabase using the Esri Data Reviewer.
• ArcFM - if using ArcFM be sure to drop the ArcFM Objects prior to your migration.
• Schema mapping: modify the schema mapping spreadsheet to match your source data and any changes you need to make to the Utility Network data model.

Review the Key Input & Output Datasets

• Input datasets
  • Source Geodatabase(File or ArcSDE)
  • Schema Mapping Spreadsheet (Microsoft Excel)
    • Asset Group/Asset Type Assignment
    • Domain Mapping
    • Attribute Mapping
  • Assembly Builder XML Template (Basic Representations Only)
  • Asset Package Geodatabase Template
• Output datasets
  • File Geodatabase Asset Package
  • Error Geodatabase
    • Redline File Geodatabase provided with error features

Simple Migration

This section summarizes the steps to modify the migration workspace.For full details,please refer to the migration documentation provided with the Migration Tools.

Understanding the Migration Workspace

The Simple Migration Workspaces are divided into several fundamental sections that are indicated by bookmarks in the workspace:

• Reader Feature Types
• Schema Mapping
  • Asset Group and Asset Type Assignment
  • Domain Mapping
  • Attribute Mapping
• Associations,例如:
  • Pump Station containment
  • Casing containment
• Special Cases
• Validation and Reporting
• Writer Feature Types

Customizing the Migration Workspace

You will need to customize the migration workspace to suite your specific source data and any customization you make of the asset package.the following sections outline how to modify the workspaces.

• Update Schema Mapper Excel File
  See Schema Mapper Spreadsheet section of the Domain specific documentation.
  • Update the Asset Group/Asset Type Assignment Sheet
  • Update the Domain/Subtype Mapping Sheet
  • Update the Attribute Mapping Sheet

NOTE: Attributes are Case Sensitive!!

• Update the Geodatabase Reader
  See Update Source Feature types section of the Domain specific documentation.
  • Add a new Esri Geodatabase Reader (ArcSDE Geodb or File Geodb)
  • Import Feature Classes from Source ArcGIS Geometric Network Geodatabase
  • Change the SourceDataset_GEODATABASE_FILE parameter to the Source Geodatabase
  • If any additional Feature Classes need to be added,use the Import/Update Feature Type function.
• Workspace modification
  • Prepare the Asset Package via Change Asset Package Spatial Reference tool
    See the Asset Package section of the Domain specific documentation.
  • Asset Package Updates (New Domains or Attributes added - clarify if this is done via Asset Package,or done via the UN and then export Asset Package!)
  • Update Writer Feature Types (if changes were made to the default Asset Package,Writer Feature Types will be required to be updated to import the updated schema)
    See Update Target Feature Types section in the Domain specific documentation.
  • Update Writer Geodatabase Template with updated Asset Package.
• 工作的结果
  • Asset Package File Geodatabase
    • Feature Classes (Devices,Lines,Assemblies,etc.)
    • C_Associations Table
• Error Report
  • Redline Geodatabase
    The Redline Geodatabase will write out features that errored out through the translation,this will allow you to visualize the problem feature and determine why it failed.The Redline Geodatabase may need to be altered to write out your desired attributes based on your mappings.
    • Feature Class(Point,Line,Polygon)
    • Subtype
    • Source ObjectID
    • GlobalID
    • Error Message
    • Error Transformer(currently only Electric workspace)
  • Excel File
    The Excel file generates a similar report to the Redline Geodatabase,without the spatial component.This can be used to compare against your Schema Mapping Spreadsheet to determine if there is a problem with one of your mappings.The Excel file may need to be altered to write out your desired attributes based on your mappings.
• More Debugging Tips n' Tricks
  • Reducing the amount of data (testing specific features)
  • Using Feature Caching
  • Using Data Inspector

Post-migration

After a successful translation to the Target Asset Package,follow the remaining steps as outlined in the Domain specific documentation provided by Esri.

Note:Using {} - GUID vs GlobalID

• FME is unable to write GlobalIDs,due to the process of enabling GlobalIDs on a Feature Class.As such,FME will write to a GUID field in the Utility Network Asset Package.Esri has tools designed to migrate these GUIDs to a proper GlobalID data type

Course Materials & Resources

Course materials including example workspaces,example datasets and slides arehere.The materials contain:

• Simple Migration: examples using Water migration workspaces & sample data
• Basic Migration: examples using Electric migration workspaces,sample data and some guidance slides
• 草案文档电迁移

Please note that the electric migration workspaces used in these examples are 'beta' release only and should not be used in a production environment.

Introduction

The Utility Network is a newly released data model developed by Esri for modeling utilities and telecoms.The new data model will allow users to more completely model their complex networks - this includes complex entities such as a Switchgear for Electrical utilities.

More information regarding the Esri Utility Network refer to the ArcGIS ProWhat is utility network?Information on the utility network asset package and how to download the can be found on the Esri ArcGISUtility Network Package ToolsGet Started page.

If you are interested in learning more about the FME Utility Network migration workspaces,please review FME Training for Esri Utility Network Migration.

FME Migration Tools

FME Workspaces are available to help migrate your Esri ArcGIS Geodatabase to the new Utility Network (UN) data model.Thestatus of the migration toolsfor the different UN data models is documentedhere.

Esri Utility Network Migration Tools include:

Gas - Migration Tools

The Gas Utility Network Migration Tools help you transform gas data from a geometric network to a utility network using FME or ArcGIS Pro with the Data Interoperability Extension.It includes a workbench,aschema mapper,tools,sample data,and documentation.

The provided workbench and schema mapper are preconfigured for the included sample source data.You will need to configure the workbench and schema mapper for your source data.

Water - Migration Tools

The Water Distribution Utility Network Migration Tools help you transform water distribution system data from a geometric network to a utility network using FME or ArcGIS Pro with the Data Interoperability Extension.It includes a workbench,a schema mapper,tools,sample data,and documentation.

The provided workbench,schema mapper,and sample source data are set up for a schema that is very similar to how a water distribution system was represented as a geometric network in the Local Government Information Model (LGIM) or the water editing tools.You will need to further configure the workbench and schema mapper for your source data.

Electric - Migration Tools

In Development.The electric migration is classed as a 'Basic' migration and includes the creation of assemblies for devices such as Fuses,Transformers,Switches / Reclosers,Switchgear,etc.

If you have additional questions or would like a more in-depth walk through of the Migration workbenches,please feel free to contact us at info@亚搏在线safe.com.