span8
span4
span8
span4
http://www.drivebc.ca/api/events/region/mainland?format=rss
Exercise 5:ex5-jsontemplater-vantraffic-xml2json.fmwt
Output file:ex5-jsontemplater-vantraffic-output.json
In the previous article,JSON Writing - Overview,we took a closer look at the data structure of the live traffic data from DriveBC and entered the URL as the data set in the reader.The output file was displayed with the Data Inspector.We used a reader in GeoRSS/RSS format to get to the features of interest.
This article will expand on that same data set of live traffic updates in British Columbia by using the JSONTemplater to combine attributes that we are reading with the XML reader on two different levels.This allows us to write a text file in JSON format that can be used for more analysis or display.Our goal is to show how to access the more nested attributes.We make no disclaimer as to the availability of this data as it changes continuously.
This simple workspace consists of two XML readers and a Text File writer.We generate the root content from the channel feature reader "rss/channel" and the sub content from the"rss/channel/item" feature reader.The JSONTemplater and JSONFormatter will be used to transform the data.The output data file is a text file in JSON format that stores the data of interest.
FME uses a template approach to writing JSON.A template represents the structure of the data,and then fme:get-attribute functions,fme:get-json-attributes or fme:process-features are placed within the template to merge in attribute values from FME features.The JSON document can be produced one per feature or by root and sub-templates.Sub-templates are used to model a document that has a root with multiple child elements based on a multi-record dataset.
With this data set,we will teach you how to generate a root document and a sub-template document.The feature types of interest can be selected by setting the Elements to Match in the Feature Path Configuration of the XML Reader.
Instead of using a JSON writer,select a Text File writing format and specify JSON as the output format.The reason the Text File writer is used and not the JSON writer is that currently the JSON writer only supports simple,flat structures.
1.Open FME Workbench.
Create a new workspace by starting with a blank workspace.
2.Add an XML Reader.
The first step for converting this data set to JSON is to generate the root element.In order to create the JSON template,we add an XML reader and enter the Dataset URL http://www.drivebc.ca/api/events/region/mainland?format=rss
XML reader parameters
3.Reader Configuration in the first XML Reader.
In Feature Type Parameters open the tree control by clicking the ellipsis [...],then click in Feature Paths Configuration and Elements to Match again the [...] button.The Select Elements to Match dialog opens and we will select the attribute "rss/channel" as the root.This selection will appear at the bottom in Selected items.This will provide the workspace with channel feature for the root template.
XML reader configuration tree
Click ok three times and your first Feature Reader will be placed in your workspace.
4.Set Feature Path Configuration in the second XML reader.
Add a second XML reader,enter as data set again the same URL http://www.drivebc.ca/api/events/region/mainland?format=rss.In Feature Type Parameters open the tree control by clicking the ellipsis [...] and in Feature Paths Configuration and Elements to Match again the [...].Then select the "rss/channel/item" and pay attention to the Selected items box,where you may want to copy the tree structure for later.This will provide the workspace with item features for the sub-template.If in the Selected Items box rss/channel and rss/channel/items both appear,uncheck the box next to channel.We are only interested in rss/channel/items for this reader.
Reader configuration tree of second XML reader
Click ok 3 times and your item feature type will open in your workspace.
5.JSONTemplater root template configuration
Add a JSONTemplater transformer and connect the channel feature type to the JSONTemplater Root input port.This will populate a JSON document with FME feature attribute values from the channel reader.In the parameters under "port" and "ROOT",click on the ellipsis button [...] in the field next to it and a new dialog will open called ROOT template expression.Here you can build up the JSON data structure.Notice the attributes listed on the left match your input data.Look for the XQuery Functions "get-attribute" and "process-features".
JSONTemplater parameters
Type the following expression in the window on the right.Click ok twice.注意,属性设置为_result结果。
Sample JSON root template with insert statements for the FME fields
Let's test this before we go any further to see how the JSONTemplater works with just a root template.Delete the { fme:process-features("SUB")} expression from the root template.Add a Logger after the JSONTemplater and run the workspace with full inspection to see what the output of the root template is.When you double click the resulting feature bubble,and Data Inspector opens,you should see the following output next to the attribute "_result":
{"title": "DriveBC Events","link" : "http://www.drivebc.ca","description" : "DriveBC Events","features" : [ ] }}
Output data structure for attribute "_result"
Now add back in the { fme:process-features("SUB")} expression to the root template and proceed with the following sub-template step.Disconnect the logger before continuing.
6.JSONTemplater sub-template configuration
First,connect the item feature type with the JSONTemplater - Root input in order to be able to select attributes.Under JSONTemplater parameters,click on Sub Template and then the + sign to add a new Sub Template Expression.This will add a new input port to the JSONTemplater.Click on the ellipsis button [...] and enter in this template structure.
JSON Subtemplate defines JSON objects with insert statements
Click ok twice and then change the connection of the item feature type from Root to the Sub Input port that is now available.Note that if you make a mistake FME will generate warnings about attributes not found,etc.
7.Add a JSONFormatter and enter the following parameters
Use format type "Pretty Print".This can help make the document more readable and easier to debug when there are problems.It also functions as a JSON validator since invalid JSON cannot be formatted and will generate an error.Under input settings and JSON Document select "_result".The output settings Result Attribute should be set to"text_line_data".Now press ok to accept the changes in the JSONFormatter transformer.
JSONFormatter settings
8.Add a Text File writer and save to VanTraffic_output.json.
Your completed workspace will now look like this.Save and Run this workspace.You can also download thecompleted workspace.
Completed Workspace
9.Open the text_line JSON output file
Navigate to open containing folder and openyouroutput.json.You can access nested JSON objects by using the dot notation or bracket notation.Examine the data document with bracket notation and the tabular format in dot notation.
{ "title" : "DriveBC Events","link" : "http://www.drivebc.ca","description" : "DriveBC Events","features" : [ {"geometry" : { "type" : "Point","coordinates" : [ "-124.562445","49.888015" ]},"properties" : { "Id" : "http://www.drivebc.ca/~RTMC_13103","description" : "Highway 101 Northbound will be closed between Joncas Pl and Sutherland Ave due to Special Event.Special Event will be in effect 19/05/2017 at 9:15 AM until 9:45 AM.Expect delays.Updated on Wed Apr 5 3:57 pm PDT.(ID# RTMC_13103)","pubDate" : "Wed,05 Apr 2017 15:57:13 -0800","severity" : "Major" }}]}
Data structure of JSON output file
geometry.type | geometry.coordinates | properties.Id | properties.description | properties.pubDate | properties.severity |
Point | -124.562445","49.888015 | http://www.drivebc.ca/~RTMC_13103 | Highway 101 Northbound will be closed between Joncas Pl and Sutherland Ave due to Special Event.Special Event will be in effect 19/05/2017 at 9:15 AM until 9:45 AM.Expect delays.Updated on Wed Apr 5 3:57 pm PDT.(ID# RTMC_13103) | Wed,05 Apr 2017 15:57:13 -0800 | Major |
The same data structure example in tabular format
10.Expected output
Open the Data Inspector and click File > Open Dataset.Change the Format to JSON and browse to the file we just created.Open up the parameters,set the Schema Scan Mode to JSON Query.Then click the ellipsis [...] next to JSON Query for Feature Objects to open up the tree.In the tree click json,features,then
JSON Output file viewed in Data Inspector
This exercise showed how a live data set from a URL can be transformed to a JSON file as output.First,the data is read by two XML readers to combine attributes from both in the JSONTemplater and then the output file is set to a text file.The output text file is written in JSON format.We wanted to demonstrate that the JSONTemplater populates a JSON document with FME feature attribute values,which is very similar to how the XMLTemplater works.You can see that the JSONTemplater gives you total control over the schema.The JSON Formatter is added after the JSONTemplater to select the _result statement as the input and connected to the text_line_data writer.The resulting JSON file can be used for further analysis.
Getting Started with JSON
JSON Reader Configuration
Converting from JSON to a spatial format (GIS)
JSON Reading Advanced
© 2019 亚搏在线Safe Software Inc |Legal