I'm reading an ArcSDE polygon layer (British National Grid),running it through the Aggregator,then writing to a TDE.But I get a warning in the log "TDE does not support geometry on feature".I can't see much in the way of parameters to change to try and fix this.有什么想法吗？FME 2018.0.1.0 (20180518 - Build 18310 - WIN64)

Beginning in Tableau version v10.5 (January 2018),extracts use the超级format instead of theTDEformat.Extracts in the .hyper format take advantage of an improved data engine,它支持与之前的数据引擎相同的快速分析和查询性能，but for even larger extracts.
Suggest creating a writer for Tableau's new extract file format "超级“。

你好，

I am trying to do a spatialJoin between a records table with lat,lon coordinates that lives on AWS Redshift,and a shapefile that lives on my local PC.

The final goal is for this join to then be imported into tableau and visualize the data as a choropleth map showing number of records per shape.

然而，for some reason this is failing for me and I have no idea why.

Below is the ETL diagram:

As you can see no output is going to inspector called PointsOnShape - suggesting that no point found any match with the shapes.I have plot the points and shapes on tableau and there's definitely overlap.

ShapeOnPoints Inspector shows also no overlap,and gives an error:

另一方面，拒绝显示一个明显的错误：无效的_点_几何_顶点。But I have no idea how to handle it or what I may be doing wrong.

This is just part of my ETL 亚搏在线workflow because ideally I'd write this data to redshift and then consume it in Tableau but wanted to start by fixing this issue.

Thanks!

介绍

Data visualization tools and dashboards are helpful for generating useful intelligence from raw data.Automating data preparation tasks is the first step to leveraging these applications.Learn how to restructure attributes and attribute values,以及如何在FME中转换数据和使用数据透视表。然后，the data is ready to load into a business intelligence tool such as Tableau.

练习

Data Validation and Cleanup

Learn to extract coordinate data from a spreadsheet and then clean up the data before loading it into Tableau.

Transposing Data for Business Intelligence

We will start with a large Excel document containing population statistics.The data will be transposed to restructure it,并选择感兴趣的统计数据

Pivot Tables and FME

Using FME transformers to construct a pivot-table is demonstrated in this example.Both the option to use the AttributePivoter transformer and the StatisticsCalculator transformer is exemplified.

介绍

This six part tutorial will show you how to clean up data and write it to Tableau,导入多个电子表格并将其写入单个或多个Tableau文件，how to reproject and overlay data,and then finally,use the newly created Tableau datasets to create density maps and spider diagrams in Tableau.For a better overall understanding of how FME and Tableau work together,follow the tutorials in order as they increase in complexity with each tutorial.

练习

How to Write Spatial Data into Tableau with FME

Work through a simple translation to familiarize yourself with FME and Tableau

How to Prepare Data for Tableau with FME (Processing Spreadsheets)

Learn to extract coordinate data from a spreadsheet and then clean up the data before loading it into Tableau

How to Prepare Data for Tableau with FME (Multiple Spreadsheets)

Multiple spreadsheets?没问题。Using the workspace from the previous tutorial,添加多个电子表格并写入单个Tableau文件或多个Tableau文件。

Reprojection and Spatial Overlay with FME

Datasets can come in any coordinate system,learn to reproject them to the same coordinate system and then overlay them on top of each other.

Creating Density Maps for Tableau

With the data cleaned up,we can now create impactful maps to share with others inside of Tableau.This tutorial you will create three different types of density maps;HexBin,Grid and Dot Grid maps.

Creating Spider Diagrams for Tableau

蜘蛛图是一种很好的方法来可视化多个点到一个点之间的距离。了解什么是Voronoi图以及如何在FME和Tableau中创建蜘蛛图。

下载

样本数据,completed workspace templates and Tableau templates can be downloaded在这里，you can also download the data from each of the individual exercises.Please unzip the files into your C:\ drive.Each exercise has a list of applicable data and templates that will be needed to complete it.

介绍

Spider Diagrams,also known as Spider Maps,are a great way to visualize lots of data that relates to a single point.In this exercise,we will be looking at the distance between businesses and their closest rapid transit station in the downtown core.We will clean up data that has extra attributes we aren't interested in and then relate the two datasets and write it out to Tableau format.最后，we will open our final dataset in Tableau and style it to create a Spider Diagram to share with our colleagues.

下载

business-licenses.csv

rapid-transit-stations.kmz

exercise5-spiderdiagram.fmwt

exercise5-tableau.zip

分步说明

1。Open a blank workspace and add a CSV reader

In the black workspace,add a CSV reader and add the business_licenses.csv file.Open up the parameters and under attributes change Attribute Definition to Manual.Change the Latitude and Longitude to Y and X.Set the Coord.System to LL84 and click ok.

2。Inspect the data

Open up the business_licenses.csv in the Data Inspector.We are only interested in the businesses which are in the downtown core,so we need to determine which attribute we can filter by.It looks like we have an attribute called LocalArea,that might contain the data we are looking for.Click on a point in the middle of the dense cluster of points near the top of the map,this is the downtown area.Looking at LocalArea,it has a value of 02-Central Business/Downtown.This is what we will filter by.

三。Add a Tester to filter by Downtown

Back in the workspace,add a Tester transformer to our CSV file.In the Tester enter: LocalArea contains Downtown and then click ok.

添加测试仪，set it to LocalArea Contains Downtown

4。Remove unnecessary attributes

The business_licenses.csv contains a lot of data that we don't need to create our spider diagram,so we should tidy up the dataset before we continue.Add an AttributeKeepr to the Passed Output port on the Tester.Open up the parameters and under Attributes to Keep,只检查businessname，BusinessTradeName,商业风格，and BusinessSubType and then click ok.We don't need to keep LocalArea because we've already filtered what we needed from it.Click ok.

5.Filter out rows without a latitude and longitude

The CSV is almost tidy,我们只需要再做一件事。Add a GeometryFilter to filter out any rows that don't contain any latitude or longitude values.Set the Geometries Types to Filter to Point.

6。Inspect the data

Add an Inspector to the Point Output port on the GeometryFilter.Open up the data in the Data Inspector and confirm that there are only points in the downtown area,that there are only 4 attributes starting with "Business" and that all values have a point on the map.

7。Create a unique ID

For the spider diagram to be compatible with Tableau,the points data requires a unique identifier.Add a UUIDGenerator and connect it to the Point Output port on the GeometryFilter.

8。Add a KML Reader

添加阅读器，enter the following:

单击确定，when the Select Feature Type dialog appears,deselect everything except for Rapid Transit Stations.

9.Change STATION to StationName

To keep with our camelcase naming style,add another AttributeManager to Rapid Transit Stations.Change the input name of STATION to the output of StationName.

10。VoronoiDiagrammer

We want to connect each business to the closest transit station.We could do this using something like a Bufferer transformer,但让我们看看VoronoiDiagrammer.The VoronoiDiagrammer creates a set of polygons showing the area that is closest to a particular input point.

Add the VoronoiDiagrammer and connect the Points Input port to the Output port on the AttributeManager connected to the Rapid Transit Stations.Then attach an Inspector to the VoronoiPolygons Output port.

11。Inspect the data

Inspecting the data,我们可以看到Voronoidiagrammer工作并创建了多边形，但它移除了我们的快速中转站。

The Voronoi Diagram in Data Inspector,notice that there are no latitude or longitude values in the Table View

12。Add a CoordinateExtractor

我们希望保留我们的快速中转站的原始点数据，so let's add a CoordinateExtractor in between the AttributeManager and the VoronoiDiagrammer.The CoordinateExtractor will store the latitude and longitude in an attribute.Open up the parameters and change the Mode to Specify Coordinate.Then change the X Attribute to trainStation_xcoord and the Y Attribute to trainStation_ycoord.Remove the Z Attribute and click ok.

In the CoordinateExtractor set the Mode to Specify Coordinate and change the X and Y Attributes

13。Relate the two datasets

Time to relate the two datasets,add the PointOnAreaOverlayer,connect the UUIDGenerator to the Points Input port and the VoronoiDiagrammer Voronoi Polygons Output port to the Areas Input port.

14。Determine which points overlap with the areas

Add another Tester and connect it to the Point Output port on the PointOnAreaOverlayer.Set the Tester to _overlaps > 0 to test to see which of the points overlaps with the polygons.

15。创建蜘蛛图

Now that we have processed our data let's turn it into a spider diagram.Add a VertexCreator and connect it to the Passed Output port on the Tester.Open up the parameters and set Mode to Add Point.Set the X Value to trainStation_xcoord and the Y Value to trainStation_ycoord.This will create a line that starts at the business point location and ends at the transit station.This is done by using the latitude and longitude attributes we created with the CoordinateExtractor.

In the VertexCreator set the X and Y Values to trainStation_xcoord and _ycoord

16。Determine the distance between the business and the nearest train station

To find the distance (as the crow flies) between the business and the nearest train station we need to measure the length of the lines.Add a LengthCalculator,打开参数并将乘数更改为100，this will give us the approximate distance in kilometers.Connect this to the Output port on the VertexCreator.Connect an Inspector to the Output port of the LengthCalculator to ensure that the translation worked before we write it to Tableau.

17。Write to Tableau

Add a Tableau writer and write it to your Output folder.Change the Add Feature Type(s) to Automatic...然后单击确定。When the Feature Type dialog box opens,change the Table Name to SpiderDiagram and change the Geometry to tde_line.运行翻译并在Tableau中查看结果。

18。Create the spider diagram in Tableau

要在Tableau中创建蜘蛛图，we need to add the spatial_latitude and spatial_longitude to the shelf.Add both the _uuid and spatial_geometry_order to Detail and then change the Mark Type to Line to see the results.最后，customize and style your map as desired.

Create a spider diagram in Tableau by adding the _uuid and spatial_geometry_order to Detail

介绍

Density Maps are a great way to visualize data quickly.In this exercise we will go over how to create three different kinds of density maps;HexBin,网格，点网格。All three of these map styles combine similar points into a bin.We will create all three maps in one workspace using bookmarks to toggle each 亚搏在线workflow on and off.

下载

Source data:business-licenses.csv

Complete workspace:exercise4-densitymaps.fmwt

表格模板：exercise4-tableau.zip

分步说明

准备数据

1。Open a blank workspace and add a CSV reader

Add a CSV reader with the following input:

Click parameters,change AttributeDefinition to manual,set the Longitude to the x_coord and the Latitude to the y_coord.单击确定，then set the Coord.系统到LL84.Click ok to add the reader to the workspace.

2。转为UTM83-10

Add a Reprojector transformer,and set the Destination Coordinate System: to UTM83-10.Connect it to the CSV Reader.

三。Remove null geometry

Add the GeometryFilter to remove any points that do not have a latitude and longitude.Set the Geometries Types to Filter to Point.将此连接到转发器。

Part 1: Hex Bins

4。Add the HexBinner Transformer

The希宾纳transformer is a custom transformer.It creates hexagonal features that enclose point feature input.They aggregate data into a grid and can be used for further analysis.

We are going to create a HexBin that is one kilometer in size.Add the HexBinner to the workspace,you will notice that it is green,while the other transformers we've seen before have been blue.When a transformer is green it means it is a custom transformer.In the parameters for the HexBinner,change the Tile Size to 1,and ensure that the Units are set to Kilometers.Connect the Input port to the Point Output port on the GeometryFilter.

Set the Tile Size to 1 and ensure the units are set to Kilometers

5.检查结果

将检测程序连接到HexBinner上的输出端口。In the Data Inspector,you should see a grid of hexagons.Clicking on a hexagon,you can see how many business licenses are in each section.

Inspect the HexBinner Transformer in the Data Inspector.All the points appear grouped in hexagons

6。确定每个箱子中有哪些业务

We would like to determine which businesses are in each bin,not just a count.To do this we will add the Counter transformer.For the Count Output Attribute,name it HexBinId,then accept the defaults for the rest of the parameters.Click ok.Connect the Counter to the HexBinner Output port.

Now add the PointOnAreaOverlayer to add the HexBinId to each of the points.Connect the Counter Output port to the Area Input port on the PointOnAreaOverlayer.Then Connect the GeometryFilter Point Output port to the PointOnAreaOverlayer Point Input port.

7。写信给

Add a Tableau Data Extract (TDE) writer and set it to your Outputs folder,also set the Feature Type Definition to Automatic...

Change the Feature Type properties,name the table HexBins,and the Geometry to tde_polygon.Click ok.

Connect this writer to the Output port on the Counter Transformer.

Then right click on the canvas click "Insert Writer Feature Type...".Add another writer for the point data.Name this one Businesses and change the geometry to tde_point.Connect this writer feature type to the PointOnAreaOverlayer Point port.最后，to tidy up the workspace and allow us to enable/disable this translation,add a bookmark from Hexbinner onwards.要创建书签，select everything you would like to add to the bookmark and then right click on the workspace and click Insert Bookmark.

Final translation layout,ensure your writers are pointed to the correct output ports

8。Run translation and view data in Tableau

While running the translation,if you encounter a Rejected Error,将记录器连接到每个拒绝输出端口，or in the Navigator > Workspace Parameters > Translation > Rejected Feature Handling,设置为继续翻译。

在表中，join together both the HexBins and Businesses tables.在你的床单里，double click on the geometry measure from the HexBins extract.Then drag _numPoints to Color to color the HexBins by the number of businesses in each.

A hex bin map completed and styled in tableau.

Part 2: Grid Map

9.Disable the HexBinner Bookmark

Right click on the Hexbinner bookmark and click disable.Now when we run our new translation,this translation branch won't run.

10。Add 2DGridAccumulator

Add the 2DGridAccumulator to the Point Output port on the GeometryFilter.In the parameters set both the Column Width (ground units) and Row Height (ground units) to 500.This will be the size of our grid.If you have time,try setting it to 250 or 750 just to see what the results look like.Then set the Type of Grid to Create to Polygons.This will create a square grid with lines.如果你有时间的话，experiment with Points (Corners) and Points (Centers).Once you have set your values,accept the defaults for everything else,然后点击确定。

Set Column Width and Row Height to 500,and set the Type of Grid to Created to Polygons

11。检查结果

Connect an inspector to the Grid Output port to view the results.In the Data Inspector under the Display Control,click the properties for the 2DGridAccumulator_Grid.Then change the Fill Opacity to 0,并将笔宽增加到3。Click ok.如果要添加背景图，click on Tools > FME Options...Then under Background Map,将背景格式更改为stamen maps。This is the free background map service that comes with FME.Then click on Parameters...to set the Map List.The example is using Terrain,but you can use the one you like the best.Click ok.

Zoom in to see the background map inside each of the individual squares of the grid.

Inspect the 2DGridAccumulator in the Data Inspector.Change the drawing styles to view the background map

Change the Pen Width to 3,and the Fill Opacity to 0

12。Determine how many points fall within each grid square

Now we need to determine how many of our business license points fall within each grid square.To do this we will add the PointOnAreaOverlayer.

Open up the parameters,let's change the name of Overlap Count Attribute to NumPoints.Click ok.Connect the Area Input port to the Grid Output port,以及指向几何过滤器上的点输出端口的点输入端口。

13。Keep only the attribute NumPoints

We are only interested in the attribute NumPoints,so we will add the AttributeKeeper transformer to the Area Output port on the second PointOnAreaOverlayer.The AttributeKeeper works just like the AttributeManager,but it is more efficient if you only want to keep certain attributes.Open up the AttributeKeeper parameters,然后在要保留的属性下，select NumPoints.

Only keep the NumPoints attribute

14。检查结果

It is a good idea to double check your results after adding or subtracting attributes to ensure that the data you want is present and you've removed the data you don't want.Add an Inspector to the Output port on the AttributeKeeper.Confirm in the Table View,唯一的属性是numpoint并且它有值。You might have to scroll to see values other than 0 since our grid covers ocean,there will be a lot of zeros.

15。Remove grid squares with zeros

正如你所看到的，there are a lot of zeros.We have no need to display those,so let's remove them to tidy up our grid.Add a Tester transformer to the Output port of the AttributeKeeper.Then set it to Numpoints > 0.Now our grid will only display squares with values other than 0.

16。Write to Tableau

最后，we need to write our grid to Tableau.Right click on the workspace canvas and click Insert Writer Feature Type,then change the Table Name to GridMap,and set the Geometry to tde_polygon.最后，connect it to the Passed Output port on the Tester transformer and run the translation.

17。添加书签

For this branch of the translation,we will only bookmark our writer called GridMap,this is because we will reuse all of the other transformers for the Dot Grid Map translation.Select the writer and either right click on the canvas to add the bookmark or use the keyboard shortcut ctrl-B .

18。View grid map in Tableau

Open up GridMap.tde in Tableau.In a new sheet,double-click on the geometry Measure to show the polygons in a map.Then drag NumPoints to Color to show the density of businesses in the area .

Grid Map styled in Tableau

Part 3: Dot Grid Map

19。Add a CenterPointReplacer

For the final map,we will be reusing all of our transformers that are not already in a bookmark.Ensure all the other bookmarks are disabled.

To create the Dot Grid Map,we only need to add a couple of things to our previous translation.Add a CenterPointReplacer transformer to the Passed Output port on the Tester.This will find the center point of the polygon square we created with the 2DGridAccumulator and turn it into a point.

Complete translation for the Dot Grid Map.Add the CenterPointReplacer after the Tester

20。Add Feature Type Writer

Right click on the canvas to Insert Writer Feature Type,then change the Table Name to DotMap,and then change the Geometry to tde_point.Click ok and run the translation.Add another bookmark containing the CenterPointReplacer and the DotMap writer.

21。Open the DotMap in Tableau

Open up GridMap.tde in Tableau.In a new sheet,double-click on the geometry to create a map.Add NumPoints to Size.On the sidebar,a legend with the sizes appears.Double-click it to open the size properties,increase the minimum and maximum dot size to increase the exaggeration.改变点地图的外观,create a color grouping for NumPoints then add it to Colors.

Dot Grid map styled in Tableau.Map shows both the color and size scale for the number of businesses

介绍

在这个演示中，we will take a large dataset,transpose the attributes then create a new output with a selection of attributes.We will start with a large Excel document containing population statistics from the City of Vancouver,and transpose the data to restructure it.因为我们只对人口总数感兴趣，we will select only the statistics of interest.Then we will rename the attributes and write the data to CSV.

下载

Source Excel data

Completed workspace

Optional data for overlay:local-area-boundary-shp.zip

来源

An excel file containing 2011 population census statistics from the City of Vancouver.

CensusLocalAreaProfiles 2011.xls版viewed in Excel.

亚搏在线工作流步骤

1。Read in the source data,CensusLocalAreaProfiles2011.xls,using the Excel Reader.Under参数，设置字段名行to 5 and单元范围到6：。

Excel Reader parameters.

2。Inspect the data with the Data Inspector.We want to make the columns into rows to make the data easier to work with.In this example,we are only interested in thepopulation totals，不是一切，因为这将是大量的数据。

Excel file viewed in Data Inspector,with one population total highlighted.

三。To get only the total statistics,add a Tester transformer.Set the测试子句到：

Tester parameters to filter out the statistics totals.

4。To the Tester Passed port attach anAttributeExploder.Open up the parameters for AttributeExploder,change Keep Attributes to Yes.The AttributeExploder will help us turn the attribute names (columns) into attribute values (rows).它使用一对新属性（属性名/属性值）为电子表格中的每个属性值创建一个功能。Attribute names become attribute values.

5.Add an AttributeKeeper,as we only want to keep a selection of attributes.Set Attributes to Keep to一个，which represents statistic type,商标名，which represents the Vancouver area boundaries,andα值，which represents the population totals.

6。Another Tester is used to filter out the unwanted attribute values such as format attributes.

Test Clauses:

7。Connect an AttributeManager to the Failed output port of the Tester.The attributes are given more descriptive names in the AttributeManager:

一个is renamed to状态类型as it represents statistic type

商标名is renamed toAreaNameas it represents the Vancouver area boundaries

α值is renamed to状态值因为它代表了人口总数

AttributeManager parameters.

8。The AttributeFilter filters attributes byAreaName.为了Possible Attribute Values，请加CMA of VancouverandVancouver CSD (City).This will filter out statistics from the whole city of Vancouver,as we are only interested in Vancouver area boundaries.

AttributeFilter parameters.

9.Add a CSV Writer,specifying an output dataset location and settingCSV File Definition到自动的.Attach the writer feature type to the AttributeFilter 端口。Rename the writer feature type to something more descriptive and ensure the attributes include StatType,AreaName and StatValue.Once the workspace is run,the output CSV file can be added into Tableau.或者，write out to the format of your choice.Feel free to overlay the data with local_area_boundary.shp.

Output CSV file viewed in Data Inspector

I would like to know if anyone has published a data source to Tableau Server from FME

介绍

数据以各种投影形式出现，有时甚至缺少投影。This is where the Reprojector transformer comes in.It transforms data from one coordinate system to another.When your datasets are in the same coordinate system they can be overlaid on top of another.In this exercise,we will take two datasets,each with a different coordinate system and reproject them into the same one (UTM83-10).Then write the output to Tableau.

下载

Source data:

business-licenses.csv

邮编

Complete workspace:exercise3-reprojection.fmwt

Tableau template: exercise3-tableau.zip

分步说明

1。Set up CSV reader to read the business license points

Click Reader > Add Reader then enter the following:

Open up the parameters,set Feature Type Name(s) to From File Name(s) to create a feature type in the workspace that gets its name from the file name instead of calling it CSV.

Set Attribute Definition to Manual then set the Latitude to y_coordinate and the Longitude to x_coordinate.Click ok.Finally set the Coord.系统到LL84.Click ok to add the reader to the workspace.

2。Add AutoCAD Reader to read the Land Parcels dataset

Add a Reader then start typing DWG into the format and select the following:

Under parameters,make sure Group Entities By is set to Layer Name,click ok and then again to add the reader.The Select Feature Types dialog will appear,select only the layer called ParcelLines.

三。Inspect the data

Connect Inspector transformers to each of the reader feature types in the workspace.You can do this by right-clicking on each feature type and selecting Connect Inspector.Then run the workspace to see the data in the Data Inspector.

When you open the Data Inspector,if the Background Map is disabled from the FME Options...窗口，your data will appear as two small dots on the screen.This is because the two datasets are in different coordinate systems.You can confirm this by right-clicking on each of the two datasets in the Data Inspector,clicking Zoom to Extent,then clicking on a feature.Then in the Feature Information window,under Coordinate System you can see which coordinate system the datasets are in.如果启用了背景图，go to the FME Options...to disable it to view this phenomena,as using Background Maps will reproject the data to the Background Map coordinate system.

4。连接一个Reprojector变压器

Let's fix this problem.Connect a Reprojector transformer to the business_licenses reader feature type and set the destination coordinate system to UTM83-10.Click ok.Move the Inspector to the other side of the Reprojector and run the translation.Inspecting the data both datasets now appear in the same coordinate space.

Reprojector transformer to reproject from LL84 to UTM83-10

5.Add a GeometryFilter to clean up data

As you recall from previous exercises some of our data in business_licenses is missing latitude and longitude data.We need to filter out all of the data missing those values so we won't get errors later on in our translation.Add a GeometryFilter after the Reprojector.Under "Geometries Type to Filter:" Click Point.Click Ok to add the transformer.

6。Determine which land parcel each business falls inside

We now want to figure out which land parcel each business falls inside.为了这个，we can use the PointOnAreaOverlayer.Add the PointOnAreaOverlayer transformer to the workspace.Connect the GeometryFilter Points Port to the Points input port and the AutoCAD reader to the Area input port.Connect inspectors to each output port.

7。Handling rejected features

Looks like all the land parcel data was sent out to the rejected port.Double click on the rejected port to open up the Data Inspector.Take a look at the Feature Information window and look for the fme_rejection_code attribute to see why the features were rejected.

The code says: INVALID_POLYGON_GEOMETRY_TYPE.It looks like we have lines from the ParcelLines dataset and not polygons.

In the Data Inspector,fme_rejection_code(string) = INVALID_POINT_GEOMETRY_VERTICES

8。Create polygons from lines

Back in the workspace,connect an AreaBuilder transformer to the AutoCAD reader.This transformer takes connected lines and turns them into solid polygons.Open up the parameters and under Snapping Pre-Processing Parameters,set Snapping Type to End Point Snapping and the Snapping Tolerance to 0.1.This will snap only the end points of lines together to create the polygon.

Set Snapping Pre-Processing Parameters in the AreaBuilder transformer

9.Rerun and inspect the workspace

Rerun the workspace and inspect the output from the PointOnAreaOverlayer.如果没有任何拒绝代码，它应该看起来更好。

10。Determine which businesses only overlap with the land parcels

Add a TestFilter to the Points output port on the PointOnAreaOverlayer.Open up the parameters and set the test condition to find where _overlaps >= 1.Click Ok to add the transformer.

Set the Test Conditions in the TestFilter to _overlaps >= 1

11。创建一个ID属性以帮助在Tableau中进行连接

We want to be able to create a visualization within Tableau that links the two datasets together so that we can click on a parcel and see only the business license data for the businesses that fall within that area.We can do this by creating an ID attribute on the areas and adding that attribute to each of the points inside the area.

Add a UUIDGenerator transformer between the AreaBuilder and the PointOnAreaOverlayer.Open the parameters and set the New UUID Name to ParcelID.The PointOnAreaOverlayer will automatically add that ID attribute onto each point feature that exits the transformer.

12。Write both features out to .TDE format

最后，we need to write everything out to Tableau.We will need two writers,每个数据集一个。

单击“编写器>添加编写器”

Under Add Feature Types > Table Definition select: Automatic and click ok.Set the Table Name to BusinessLicenseData and set the Geometry to tde_point,单击确定。Connect it to the output of the TestFilter (@Value(_overlaps) >= 1).Open the writer feature type parameters and clean up the attributes to remove anything we don't need and adjust the column types as needed (we don't need _overlaps,经度，or Latitude anymore)

要为地块数据创建输出，right click on the workspace and click "Add Feature Type".然后将表名设置为LandParcels，并将几何体设置为TDE_Polygon，单击确定。Connect it to the Areas output port on the PointOnAreaOverlayer transformer.

13。Run the workspace to create the TDE files

Ensure that you have removed all of the remaining inspectors,and then run the workspace to create the TDE files.Check in Windows Explorer to confirm the files have been created.Open them up in Tableau to finish.

Final output in Tableau.Create a Business License Status map or view which businesses belong in each parcel and the status of their business license.

介绍

In this exercise,we will work through setting up a simple FME Workspace to convert a MapInfo dataset containing Parks information to Tableau's TDE format.

下载

源数据集:

drinkingfountains.csv

邮编

Completed workspace:exercise1-spatialdata.fmwt

Tableau template:exercise1-tableau.zip

分步说明

Part 1: Simple Format Conversion

1。Generate a simple FME Workspace

Start FME Workbench and click on the Generate option from the Start page.

设置读写器如下：

Click OK to generate the workspace.

Generate workspace with an MITAB Reader and a TDE Writer

2。检查数据

It's probably a good idea to know what we are working with here to know what to do with our data.我们可以在我们的机器上安装mapinfo（一个gis应用程序）并直接打开其中的文件，but all we really want to do is inspect our data source here.Let's use the FME Data Inspector application to look at this MapInfo dataset.To do this click on the "Parks" Feature Type in the workspace.Then click the icon with the magnifying glass on the toolbar that appears above the feature type.

Inspect the Parks Reader

The Data Inspector application will open and show us the geometries and data inside the file we are working with.

三。清理工作区。

Since we only have area features in our data source,let's remove the GeometryFilter and three non-polygon feature types that were automatically added to our workspace.

• Delete the Geometry Filter transformer

4。Modify the schema on the Tableau writer.

Our writer is currently set up to create a file called Parks_polygon,let's change that to something nicer.Click on the cog-wheel icon (or just double click) on the writer feature type to open the properties dialog.Change the Table name parameter.让我们把它设置为“公园”。

Workspace cleaned up with Reader and Writer renamed

5.Save the Workspace.

Click File > Save As to save a copy of the 亚搏在线workflow we have built so far.

6。Run the Workspace.

Click the green play button in the Workbench toolbar to run our data conversion to actually create our TDE file from the MapInfo dataset.

Part 2: Spatial Data Blending

We have been given a CSV file containing information about the locations of drinking fountains in the city and would like to find out how many fountains are in each park.The drinking fountain dataset does have a Location column,but on closer inspection,the location names there don't match up with the park names from our MapInfo dataset.The CSV file does have x,y coordinates in it,maybe we can join these datasets spatially.

1。Add CSV Reader into our workspace.

Click on Readers > Add Reader and set the following:

Now we will setup the parameters for the Writer.Click on Parameters...For Feature Type Name(s) select From File Name(s).Under Attributes > Attribute Definition,click on manual.Set the type for x_coord to x_coordinate and y_coord to y_coordinate.This will tell FME to display each row in the CSV as a point.单击确定。Then set the Coord.System to UTM83-10.Click Ok to add the new reader to the workspace

Add a CSV Reader to the workspace,改变坐标。System to UTM83-10

In the parameters of the CSV Reader,change the type of the x_coord and y_coord

2。Inspect the two datasets.

Disable the Tableau writer feature type for now by locating it on the canvas,right-clicking on it,and selecting "Disable".Connect an Inspector transformer to the output ports on each of the readers inside the workspace.单击运行以查看数据检查器应用程序中的两个数据集。

Workspace with the inspectors and the Parks Writer disabled

三。Use FME to spatially relate the two datasets.

Let's use a transformer to find the water fountain points that overlap with the park areas.

将PointOnAreaOverlayer转换器添加到工作区中。

Remove Inspectors from previous step and connect the Parks feature type to the Area input port and the CSV feature type to the Point input port.Then connect the Area output port to the TDE writer.

Access the PointOnAreaOverlayer transformer Parameters.It's currently set to create an attribute called _overlaps with a count of the number of features that overlap with each other.我们暂时不谈这个了。Click OK to close the parameters dialog.

4。Modify the writer feature type properties.

We want to add a new column to the TDE file we create with the count of the number of fountains that were found inside each park.Open the writer feature type properties dialog.Then open the User Attributes tab to look at the columns that are currently defined on the writer.

Add the _overlaps attribute to the list of columns there.You can do this either by switching the Attribute Definition to Automatic (to automatically pick up any new attributes added to the workspace) or by setting the Attribute Definition to Manual and manually typing in the name of the attribute we want to add.

User Attributes in the Writer parameters,add the attribute _overlaps

下一步，go to the Parameters table,under Table > General change Table Handling: to Drop and Create.This will tell FME to delete the existing TDE file and re-create it with the new column added.

Writer parameters,change the Table Handling to Drop and Create

5.Save and Run the Workspace.

Enable the Tableau writer feature type by locating it on the canvas,right-clicking on it,并选择“启用”。保存修改后的工作区，然后运行它。

6。Optional: Inspect the data in Tableau

Open up your data in Tableau and experiment with how you would like to display your data.

Parks.tde opened in Tableau.Each circle is a single park.Stanley Park has the most drinking fountains,29。

Previous Section: Processing Spreadsheets

介绍

In this exercise,we'll show how to process multiple CSV files in FME.We'll modify an existing FME workspace translating data from a single CSV file to Tableau.The workspace has been set up to process business license data from a single CSV file.Data validation and cleanup is performed before the data is written to Tableau format.But,there are also a number of other CSV files containing business license data from previous years.We will set up FME to handle all of these files.然后，we will demonstrate 2 options for writing out to Tableau:

Option 1: Write all data to a single Tableau file.We will add a new field to the Tableau file which holds the year the business license data was collected.

选项2:写数据到多个表文件。A Tableau file will be created for each CSV file that is read.

下载

Source data:邮递员

Starting workspace:exercise2-startingworkspace.fmwt

Completed workspace:exercise2-multiplesheets.fmwt

分步说明

1。Open up the previous exercise and remove DataCleanup.xlsx writers

We will be using the workspace from the previous exerciseHow to prepare Data for Tableau with FME (Processing Spreadsheets).If you haven't completed the previous exercise and would like to,click on the link to go to the exercise.If you are only interested in this exercise,下载起始模板。In the previous exercise,we already created an excel file to store all our values that need to be cleaned up,delete both DataCleanup.xlsx writers.

2。Set up CSV reader to read all of the files from a folder

In the Navigator pane,expand the CSV reader.Double-click on the source CSV file parameter,然后打开高级浏览器。Click on "Select Multiple Folders/Files..." Navigate to the folder containing the four CSV files,and select it.Read the files with a .csv extension from that folder.Remove the .gz file and the .txt file,and the previous .csv file,we are only interested in .csv files from the PastYears folder.

Datasets within the PastYears folder:

• 2012business_licences.csv
• 2013business_licences.csv
• 2014business_licences.csv
• 2015business_licences.csv

Change the Source CSV file(s) in the the Navigator

Select Multiple Folders/Files...then click on the PastYears folder to add it

三。Modify source feature type

Now that we've set up the reader,the next step is to set up the existing source feature type on the canvas to handle all the files read by the CSV Reader.Open the Writer Feature Type properties.It was originally set up to read the single business license file.Checking the Merge Feature Type option allows this feature type to process all of the CSV files that are read.We'll use the default wildcard option the Merge Filter and Filter Type.It's important to note that when we turn the Merge Feature Type,FME automatically exposes an attribute called "fme_feature_type".Each feature read is tagged with this attribute,which holds the name of the file each feature was read from.We'll make use of this attribute shortly.The name of the reader will have changed from business_licenses to

Open up the properties of the writer,enable Merge Feature Type and accept the defaults

4。检查数据

确认已设置FME以处理该文件夹中的所有csv文件。Right-click on the source feature type and inspect the data.Confirm that all 4 files in that folder were indeed read.By further inspecting a single feature,we see that the name of the file (or feature type) is stored with the feature.

5.使用VertexCreator转换器创建几何点

Connect a VertexCreator transformer to the Reader.This will create points with our Latitude and Longitude attributes.For mode,ensure that "Add Point" is selected and then change your X Value to read the Longitude attribute and your Y Value to read the Latitude attribute,然后单击确定。

Add a VertexCreator and set the X and Y Value

Option 1: Write to Single Tableau File

If we ran this workspace now,all of the data would be written to a single Tableau file.在那种情况下，we should create a new attribute to store the year the business license data was collected using a SubStringExtractor.

1。Add SubStringExtractor

Place a SubstringExtractor between the AttributeManager and the writer feature type.Set it up to extract the first 4 characters of the fme_feature_type attribute,which is in effect the year.Store the year value in a new attribute called YearCollected.

The destination Tableau schema should be updated with the new attribute.

We only want the year from the fme_feature_type,set the Start Index to 0 and End to 3

2。Run the workspace

A single Tableau file is created,with a new attribute YearCollected.View this file in Tableau to ensure the YearCollected field is populated

View the BusinessLicenses.tde in Tableau to ensure the YearCollected field is populated

Option 2: Write Data to Multiple Tableau Files

It is easy to create a separate Tableau file for every year of business license data.

1。Disable SubstringExtractor

If you added the SubstringExtractor in Option 1,disable it for Option 2,by right clicking on the transformer and clicking "Disable"

2。Modify destination feature type

Open the properties of the destination schema (Writer).Click on the drop-down next to Table Name,and select fme_feature_type.We are instructing FME to use the value of this attribute for the output table name,这意味着，that for every unique value found,a separate file will be created.我们知道，fme_feature_type包含每个功能读取的文件名，and since we are reading 4 CSV files,we expect 4 tableau files to be created.

Open up the .TDE writer and change the Table Name to the attribute fme_feature_type to create 4 files

三。Run the workspace

Navigate to the Output folder to confirm the 4 tableau files were created.If running this translation multiple times to completion.In the Writer properties,under Table Settings > General,for Table Handling: Drop and Create.

Windows资源管理器中的4个Tableau文件

Next Section: Merging Multiple Spreadsheets

介绍

This tutorial consists of 2 sections:

1。How to Prepare Data for Tableau with FME (Processing Spreadsheets) (Current Article)

在第1节中，we'll process a single CSV file and perform data validation and cleanup before loading it to Tableau.

2。How to Prepare Data for Tableau with FME (Merging Multiple Spreadsheets)

在第2节中，we'll modify the FME workspace from Section 1 to handle multiple CSV files.

下载

Source data:business-licenses.csv

Complete workspace:exercise2-spreadsheets.fmwt

Tableau template:exercise2-tableau.zip

分步说明

Part 1: Simple Translation

We will start by creating a simple translation,look at the results in Tableau,then come back to FME to do the data validation and cleanup.

1。创建新工作区。

Start FME Workbench and select the New option under Create Workspace.

2。Set up CSV Reader.

第一步是读取csv文件。将业务许可证CSV文件从文件资源管理器拖到空白画布上。通知，that FME has already filled in the reader format and dataset.或者，click Add Reader.

Add a CSV Reader,add the business_licenses.csv,set the coord system,then click on Parameters...

Click on the Parameters button.The Database Parameter allows us to choose different naming schemes for the layers or feature types that end up on the canvas.确保将其设置为“文件名中的功能类型”。

We can tell FME to convert the CSV file latitude and longitude values to points AS it reads the data.要做到这一点，first double-check that the File Preview is correct,then under Attributes > Attribute Definition,click on manual.将纬度设置为y_坐标，将经度设置为x_坐标。This will tell FME to display each row in the CSV as a point.

Change the Attribute Definition to Manual and update the Latitude and Longitude

单击确定，and then click Ok again to add the Reader to the canvas.This tells FME to create a point for each record with a latitude and longitude value.

When the CSV source data is added to the canvas,click on the arrow to see the full attribute list.

三。Confirm points are created as CSV file is read

Right-click on the source feature type,and select Connect Inspector.Go ahead and run the workspace.Data is read and directed to the FME Data Inspector.We can see that we do,事实上，have points.

Business Licenses with point data in the Data Inspector

4。Add Tableau writer

From the Writers menu,select Add Writer.对于格式，start typing Tableau,and select the Tableau Data Extract format.对于数据集，选择要在其中写入.tde文件的目录。Click OK to add the Tableau writer to the workspace.Then connect the Reader and the Writer.

5.Modify writer feature type properties

Open the properties of the Writer.We can now specify the name of the table we would like to write to.Call it BusinessLicenses.

Change the Writer Feature Type Properties by changing the Table Name

6。Run workspace

You may have noticed a number of blue warnings go by in the log file.These are related to problems with the CSV data,which we will soon fix with FME.

7。(Optional) Examine output TDE file in Tableau

在表中，我们可以看到所有的列都对应于我们用FME从csv文件中读取的列。Notice that the data types of the columns have all been set automatically by FME based on the best guess at what kind of data is inside each one.We can see that the LicenseRSN has correctly been set to a Number type and BusinessName is a String.

查看Tableau中的数据以确保它设置了正确的数据类型

Part 2: Data Validation and Cleanup

Let's take a closer look at the business license data.

Here is a list of tasks we will accomplish with FME.We have already completed the first one in our current workspace.We will modify the workspace to perform the rest of the tasks.

• Read Excel file and create points
• Filter out records that don't have latitude/longitude values
• Ensure PostalCode has a value;extract first 3 characters
• Set up conditional value to handle empty FeePaid values
• Create new BusinessDisplayName field;populate from existing fields

1。Filter out records that don't have latitude/longitude values

You may recall that some of the records in the CSV file did not have latitude and longitude values.Since FME would not have been able to create points for them,we want to filter them out;a GeometryFilter will help us accomplish this.

Click on the CSV Reader and then start typing "GeometryFilter" click enter.Click enter again to enter the Transformer Parameters.Select "Point" for the "Geometry Types to Filter".单击确定。

2。Run workspace with full inspection

Go ahead and run the workspace,making sure that the Run with Full Inspection is selected.Run > Run With Full Inspection

We can confirm that 731,out of our 10,000 records did not have latitude or longitude values.

Workspace with "Run with Full Inspection" turned on to view 计数

三。Write records with no latitude/longitude to "Data Cleanup" Excel file

Before we continue processing the points,let's write these records out to a "Data Cleanup" file,so that they may be fixed.We will write them out to Excel.

From the Writers menu,add an Excel writer.Write it to the output folder and call the file DataCleanup.xlsx.Connect the new writer feature type up to the unfiltered port of the GeometryFilter,then open up the Writer's properties and change the sheet name to "Missing Latitude Longitude".

Change the name of the sheet to identify what we are recording

Now that we have dealt with the missing values,let's continue processing the points.

4。Ensure PostalCode has values: AttributeValidator

在画布上放置一个attributevalidator，并将其连接到几何过滤器。Open its properties and select PostalCode for the Attribute to Validate.The validation rule is that PostalCode MUST have a value.也，validate that some of the string fields have string values and that the numeric fields have numeric values.

Set the AttributeValidator to validate if PostalCode has a value and other attributes are the proper type

5.Write records that fail validation to new sheet in "Data Cleanup" Excel file

再一次，在继续处理有效数据之前，write the data that fails validation out to a different sheet in the "Data Cleanup" excel file already set up.Right-click on the canvas and select "Insert Writer Feature Type".Call the new sheet "Failed Validation".Connect it to the Failed port of the AttributeValidator.

6。Extract first 3 characters of PostalCode: SubStringExtractor

Now that we know that the "records output" from the AttributeValidator all have a value for Postal Code,let's extract the first 3 characters.We do this because Tableau uses the first 3 characters of the postal code to automatically map the areas.

Place a SubstringExtractor on the canvas,and configure it to extract the first 3 characters from the PostalCodeAttribute.We will call the resulting attribute PostCodeTrimmed.

FME indexes the first value as 0 then counts from there.As you can see in the table below,if we only want the first 3 characters in the postal code,we would go from 0 to 2.So in the SubstringExtractor our Start Index = 0 and our End Index = 2.

Setup the SubstringExtractor to extract the first 3 values from PostalCode

7。Set up conditional attribute values for FeePaid: AttributeManager

The AttributeManager is a transformer that allows us to do many attribute manipulations,including setting up conditional values.

为feepaid在attributeManager中创建条件值

If the FeePaid attribute is empty,set the value to 0,否则，别管它了。

Create a new attribute called BusinessDisplayName.Set its value to BusinessTradeName,but only if BusinessTradeName has a value,otherwise set it to BusinessName.

If BusinessTradeName has an attribute Value,keep it BusinessTradeName otherwise name it BusinessName

There is also an attribute we don't need in our final output,let's remove _fme_validation_message.

在属性管理器中，create 2 conditional values for FeePaid and BusinessDisplayName,also remove _fme_validation_message

Connect the AttributeManager output to the Writer feature type.

8。Modify writer feature type properties

The attributes we removed just now are still on the output schema and have turned red.The original attribute schema was a copy of the source schema.它已经改变了，as a result of our data transformation,but we may update it to reflect what we have done.

Open the properties of the destination feature type,转到“用户属性”选项卡，and click on Automatic for Attribute Definition.The attribute schema reflects the changes we have made.

Since we want to overwrite the Tableau file we initially wrote,open the properties again and change the Table Handling to Drop and Create.

9.Run workspace and confirm .tde file exists

Let's run our final workspace!Click on the Run button.Confirm the .tde file and DataCleanup.xlsx files were created using Windows Explorer.

10。(Optional) Examine .tde file in Tableau

In the "Data Source" view,notice all of the records that we have imported from the CSV file.We can also see that the PostCodeTrimmed attribute,that we created within our workspace,is present along with the other fields imported from the CSV file.

Now that our data is imported into Tableau,我们可以开始创建数据视图来研究这些数据。

例如，we can create a simple map view to see the data points overlaid on a map.创建新工作表，然后双击几何图形以查看各个点。最后，color them by status to get an overall view of which businesses currently have active licenses.

We could also make use of the postal code field to show our data,create a new sheet.Add PostalCodeTrimmed,from "Dimensions" and color them by the unique count of business licenses in the area.

在next exercise，we will modify the workspace we just built to process multiple CSV files.

Basically converting an ESRI polygon Shapefile or FeatureClass from spatial to non-spatial in a format ready for Tableau (i.e tabular like excel or CSV).Have researched and tested some possibilities but would think there are some quick start workspaces available from Tableau and FME users.

有一些成功地将多边形顶点转换为点的想法（有帮助的VertexCounter）

Other transformers helpful are DonutHoldExtractor,2DForcer,计数器推广者？，CoordinateExtractor - just hoping something already exists.

你好，

I would like to compare two attributes of the same table: e.g.具有以下属性的功能表：uid，旧名，new name.

I would like to know for which features the old name is the same as the new name and identify the features whose name has changed.

Afterwards I would like to create new tables for all (and only) these features whose name has changed,one with the UID and the old name and one with the UID and the new name.

我应该使用哪些变压器？

Thanks for you help!