span8
span4
span8
span4
This example is intended to show how a group-based (blocking/feature holding) transformer causes the flow of data to be interrupted in a workspace.
This workspace is a simple one that reads a dataset of people (in Shape format),measures their height,and writes it back to another dataset (again in Shape format):
NB:The Junction transformer exists only to help demonstrate the relevant events in the translation
The important part is that the Sorter interrupts the flow of features through the workspace,and this can be used to demonstrate some key points about that concept.To do so,carry out the following actions.
1) Open the workspace (or template).Right-click on the connection between the reader feature type People and the Junction transformer.Choose the option to Add Breakpoint.
2) Select Run > Run with breakpoints from the menubar:
Be sure that Feature Caching (Full Inspection) is turned OFF!Breakpoints and feature caching are not compatible with each other.
Now run the workspace.Notice that a single feature is read from the source data and pauses the translation at the breakpoint.A dialog opens to inspect the feature:
3) Click the "Step to next connection" button on the Feature Inspector window.This moves the feature on to the next connection (Junction -> LengthCalculator).The red line denotes this.Click the "Step to next connection" button again and the feature will emerge from the LengthCalculator:Output port.Notice that it now has a Height attribute:
4) Click the "Step to next connection" button once more.The original feature has stopped before the Sorter transformer and a second one entered the workspace.
Lesson:The Sorter transformer is a group-based transformer.It requires all of the source features to be present before it can process any data (it can't sort a single feature by itself!) and so holds those features in memory while it waits for the others to catch up.
5) Now start clicking the "Continue to next breakpoint" button on the Feature Inspector dialog.Subsequent features will be processed up until the Sorter transformer.
Lesson:Because the features being stored by the Sorter transformer are being held in memory,the amount of system resources used increases every time a new feature arrives.If the features were large enough (ours are very small) you might even see the memory use increase on your system's Task Manager.
6) Continue clicking the "Continue to next breakpoint" button on the Feature Inspector dialog until the feature count between LengthCalculator:Output and Sorter is 107 or 108:
If you don't wish to click the next button so many times,click the red square button to stop the translation.Right-click on the breakpoint on the canvas and choose Edit Breakpoint.At the foot of the edit dialog,change the Stop option from Always to When hit count is equal to 108.Now re-run the translation and it will stop at feature 108 instead of the first feature.
Lesson:This is also showing that features read into FME remain in the same order that they occupied in the source dataset.The order of features doesn't change unless forced by a transformer - like the Sorter is about to do.
7) Take a note of at what line the log window has reached.
Now click the "Step to next connection" button three times to advance the next feature through to the Sorter.Keep clicking,three clicks at a time,until the final feature emerges from the reader and causes the Sorter to sort the data.
You'll know when the final feature has been reached because the log window updates with a few more lines of information.The most important message is the line "Emptying factory pipeline",which denotes that the Shape reader has finished reading data and has closed down.
Also notice the information finally logged about the LengthCalculator (Cloned 110 input features into 110 output features),and the messages that the Sorter has started and completed sorting 110 features:
Lesson:In truth,the closing of the reader,final action of the LengthCalculator,and sorting of data each probably took place on each of the last three clicks,but the log window just took a while to catch up with what had happened.For the same reason,you might find that stopping step #6 at 107 features is better (it can happen that the feature count is lagging behind the feature that is actually being processed).
8) Click the "Step to next connection" button once more.The first sorted feature is sent to the Writer,which is only now activated.Notice that the log window shows that a Shapefile writer has been created (Opened Shape File 'C:\FMEData2017\Output\Training\SortedPeople.shp' for output).
Lesson:Given that the Reader has now closed before the Writer has opened,we can now actually write back to the same dataset being read from.It's still not recommended behaviour,though.
9) Click the "Continue translation" button to run the workspace without stopping at any more breakpoints.
Lesson:One final lesson is to take note of the peak amount of memory used by the translation.It will be logged at the foot of the log window.In this translation it shows the amount of data stored in the Sorter transformer.Any attributes that are stored on the data are accumulated in this way,whether or not they are written to the output dataset.Therefore it makes sense to use an AttributeManager at the start of any workspace,to remove excess attributes and reduce the amount of memory used in a group-based transformer.
© 2019 亚搏在线Safe Software Inc |Legal