span8
span4
span8
span4
Each FME translation is usually a single process on your computer.However, FME can be set up to take advantage of multiple-core processors and improve parallelization of computations (doing multiple tasks at once).FME also makes use of hyper-threading, a technology used to make each physical core appear as two logical processors to the host operating system.
By using parallel processing, performance may be improved significantly over a single process.
As a brief introduction, note that each parallel process in FME uses its own set of data, and data cannot be passed between processes.Therefore you must divide data into groups using a Group-By parameter, and set each group to be handled by a different process.
Here a user is calculating statistics about the number of visitors to parks in the city of Vancouver, using FME's StatisticsCalculator transformer.Each park has an attribute that defines which neighborhood it resides in.That neighborhood attribute is used to group the data and by setting a Parallel Processing Level, each group is handled by a separate process, potentially improving performance.
The processing level determines how many processes run in parallel.Minimal creates the fewest processes.Extreme creates the most.The exact amount depends on the number of cores and processors on the computer being used:
However, there is a limit to the number of processes that FME will create.This limit is tied to the FME license level.FME Base Edition allows a maximum of 4 processes;Professional Edition: 8;All other editions: 16.
Here are some general tips for parallel processing:
In theory, parallel processing should produce results faster than a single process.However, there are instances where that might not be the case:
Rather than have a single transformer carry out parallel processing, it's possible to enable parallel processing on a whole group of transformers.
This is done by creating a Custom Transformer from that group.A custom transsformer has its own parameters for parallel processing, and it does not have to be limited to a single transformer within it.
使用自定义变压器像这也意味着,在“分组依据”和“平行过程”的设置可以不同(例如我可能会组我一起公园附近通过,但并行处理这些城市的基础上)。
这是一组实施例中,其中并行处理是使用的。对于一个教程,开展自己,看this page.
这里所有的例子都在64位Windows平台上的一个四核心(8个虚拟处理器)机4GB内存进行。请记住,结果可能取决于硬件配置和FME版本而异。
因为表面建模是如此强烈的过程中,采用并行处理可以是非常有益的。
本实施例中生成从点云输入DEM:
该RasterDEMGenerator基通过被设置为fme_basename处理每个点云作为自己的组。
适度是最好的结果在这里,超过两倍的速度,因为没有并行。最小并行度比较慢,因为它不使用全部的处理功能。激进和极端的模式是慢,因为他们使用的是完全的处理能力,在每个人的费用。
在第二个实验中,点云文件按照他们的名字的第一个字母组合(这是SubstringExtractor变压器是什么)。
使用较大的测试数据集显示了结果做规模与数据大小:
TINGenerator类似于RasterDEMGenerator SurfaceModeller的另一个子集。
在该示例中单个TINGenerator需要5分钟,以产生表面。然而,即使并行之前,我们可以用一个特技其中一个TINGenerator使得小的表面(来自每个源的LAS)与第二TINGenerator到这些小的表面结合成一个单一的表面。
耦合这双曲面生成与并行处理给出了优异的结果:
基于所述测试的结果上面我们可以决定性地得出结论,并行处理允许更快的表面建模,并且可以推荐机器支撑多线程。
这个例子使用含有US主要道路shape文件数据集,其中所述意图是缓冲用25米缓冲区各道路。这一过程将包裹Bufferer变压器定制变压器的内部,使得在group-by参数可以使用不同的属性,以并行处理由参数。
因为它让我们创建组的最佳数量,这里是8和16之间,这非常有用。该循环计数:变压器是用来做这个,它的“最大计数”参数是创建组数:
对于45万个原路段的缓冲数字如下:
我们可以看到,在过去的测试组的尺寸较小不赔多道开销(发射了FME会话和FME实例之间发送功能)。这将是一个问题的少,每组有特征的数目大得多。
并行处理可以在工作区中的任何变压器一起使用。在上面的例子中,并行处理的LineJoiner变压器(在定制变压器包裹起来)给出以下:
我们可以得出这样的结论并行处理有超过正常连线没有优势。然而,当这些数据大约是5倍大,其结果是完全不同的:
没有并行处理,一个单一的过程可以养猪资源,瘫痪计算机而FME优化存储器使用和高速缓存数据到磁盘。
裁剪就是多进程是有益的其他操作。在这个例子中,我们看到美国的主要道路,被国家中已经加入,并把它们夹到县界。同样,我们使用的FIPS数量的第二位作出处理组:
大型数据集(〜45万层的功能)的结果给出如下:
以更大的数据集(〜2250000个特征)的结果更为明显:
在3D裁剪点云可以是一个简单的表面过滤有用:
如下在相对较小的点云看结果:
欲了解更多信息,请参阅Parallel Processing documentation或者Parallel Processing section of the Desktop Advanced Training。
How To Use Parallel Processing in FME
Converting Point Clouds to Surface Models Using the PointCloudLASClassifier
How to Read and Translate all Feature Classes from Multiple Esri Geodatabases
剪裁 and Tiling Point Cloud Data
In what order are features processed when there are parallel transformers
Maximum concurrent FME processes error
Creating Boundary and Point Features from a Point Cloud
If I have several datasets that get merged within a workspace how can I separate it out again
© 2020 Safe Software Inc |Legal