span8
span4
并行处理是对多个进程的工作划分。Modern computers have multiple CPU cores and spreading work across these cores makes the most of the available computing power.
Parallel processing in FME means the FME Engine divides its work into multiple worker processes.The workspace author decides when to use parallel processing and how many processes should be created.然后操作系统决定如何将资源分配给这些进程。
FME中的并行处理应用于定制变压器。(在2019年以后的FME中,it is now the only place to find parallel processing options).本质上,通过自定义转换器的每一组功能,do so in a separate process.The data combines back into a single group when it exits the custom transformer.
因为自定义转换器被视为处理任务的一个单元,the individual transformers that it contains are all included in each parallel process.这允许作者在一个设置中设置多个要并行处理的变压器。It also minimizes the number of required processes,so there is less overhead on starting and stopping processes,来回传递数据。
Parallel processing is defined by组,yet some individual transformers are feature-based,not group-based,and so unsuitable for parallel processing.然而,a custom transformer acts as a group-based wrapper,providing a way for feature-based transformers to be processed in parallel.
Each parallel process in FME uses its own group of data,其中该组由属性定义。Firstly a level of parallel processing must be selected in the Navigator window of a Custom Transformer definition:
After that the attribute to group by is selected using the transformer's parameter dialog:
这里,用户使用ZoneCategory属性定义组进行并行处理。
Follow these steps as an example of how to use parallel processing on a custom transformer.
1。启动FME Workbench并打开附加的工作区模板:
工作区正在读取一组公园(空间)数据。It calculates the average size of each park (AreaCalculator and StatisticsCalculator) using NeighborhoodName as a group-by parameter.这将创建每个社区的平均公园大小。
然后工作空间将相邻的公园多边形分解在一起,buffering them slightly to ensure adjacent parks become merged.第二个缓冲区缩小了公园的大小。
Parallel processing is possible here,where each neighborhood is processed as a separate process.This works because all Group-based transformers (StatisticsCalculator and Dissolver) use the same group.
Other transformers are feature-based and so grouping is not important.
2.选择画布上的所有五个变压器(不包括任何检测变压器)。按ctrl+t(或从菜单栏中选择“变压器>创建自定义变压器”)。
When prompted,输入一个简单的名称,如ParkProcessor…
Then click OK to close the dialog and create the custom transformer.
三。在自定义转换器定义(活动选项卡应标记为ParkProcessor)中,浏览导航器窗口以查找并行处理参数。在变压器参数下。
Double-click the parameter and select Moderate as the processing level:
单击“确定”,将打开并行处理。
4.返回主画布(活动选项卡为主)。Click on the cogwheel button to open the parameters dialog for the ParkProcessor transformer.在group by参数中选择neighhoodName作为并行处理的属性:
Now,运行时,the custom transformer will create a separate process for each neighborhood.Several processes will run in parallel (with Moderate processing,one for each CPU core).
5.Notice that,在工作区和上面的屏幕截图中,that there is already a parameter for NeighborhoodName.
回想一下,FME会自动发布自定义转换器中使用的所有属性。这使得变压器可以在多个地方使用,where (in this example) "NeighborhoodName" might not be available.So FME automatically made the first of these published parameters when the custom transformer was created,第二次激活并行处理。
我们两者都需要吗?Well,that depends where NeighborhoodName is used.Any reference to it in Group-By parameters can be removed,because the whole custom transformer is now one large group-by.
让我们把这些参考资料整理一下。Return to the custom transformer definition.
Open the StatisticsCalculator parameters dialog and unset the Group-By,by deselecting NeighborhoodName.Repeat the process for the Dissolver transformer:
检查AreaCalculator和BuffererTransformer中neighhoodName属性的其他用途。There being no uses,we can remove the user parameter that prompts for this attribute.Do this by opening the parameters dialog for the Input port,取消选中邻居名称:
This tells FME that the attribute is no longer required in the transformer (except for Parallel Processing),并且可以删除用户参数提示。
6。Check the parameters dialog for the custom transformer.Run the workspace.输出与之前相同,但是,有了并行进程,运行速度可能会快得多。
最后一点有趣的是:虽然基于特征的变压器没有组来并行处理,they are still processed in parallel.这是另一种提高性能的方法。
© 2019 亚搏在线Safe Software Inc |Legal