Docker和FME：司机和工作流程亚搏在线

The promise of Docker is that it can change how we deploy FME processing capabilities both on-premises and in the cloud.

For us, Docker is the ultimate abstraction layer in which to deploy fault-tolerant, scalable solutions. First Docker abstracted compute, then it abstracted networking, and soon it will abstract storage. As someone who did my graduate work in distributed and parallel processing, this is very exciting!Read more about Docker’s future plans around storage.

我们希望分享我们早期的码头实验，帮助我们学习和理解Docker Swarm和服务。好的事情是你可以在一台机器上尝试这个，知道它在形成群体的任何数量的机器上都有相同的工作。

If you want to perform the demo yourself you can find all resources at the end of this post.

这是FME和Docker系列中的第3部分。

• Part 1: FME Server for Docker (Technology Preview)
• Part 2: FME Server on Mac using ‘Docker for Mac’

Simple Docker Swarm Demo with FME

The demo is very simple and consists of two workflows (FME Workspaces): a driver workflow and a worker workflow (10 engines). This is shown below.

观看Docker Swarm和FME的视频演示。

司机工作流程亚搏在线

这个FME工作区只读取一个CSV文件的城市名称。对于每个城市的名字，它都会将任务提交给一个码头工人群。工人们生成一个网页，显示城市的空气质量。驱动程序不等待任务完成，而是简单地向工作人员提交所有请求。所有作业都是通过TCP/IP上的单个端口提交的。对于驱动程序，它只是向“DockerSwarm”计算结构提交请求。该驱动程序不关心甚至不知道群的计算能力。事实上，司机甚至不知道是在跟一群人说话。就驱动程序而言，它只是向单个独立FME工作人员提交一份工作！对于这个演示，我们正在Docker之外运行驱动程序。它认为它只是与一个FME引擎通信！那多酷啊！！

Docker Magic在哪里？

这里的神奇之处在于，就驱动程序而言，它只是向单个工人FME流程提交作业。它连接并使用一个TCP/IP端口，在其中发送所有请求。

司机完全忘记了有多少工人弥补了群体，或者甚至甚至与蜂拥而至。Docker Swarm神奇地照顾网络与工人与工人共享连接的细节，无论工作人员是否在单个机器上或跨多台机器传播。Docker Swarm只是隐藏和处理所有丑陋的细节。

工作流程亚搏在线

工作流程是一个类似于我亚搏在线们在期间使用的FME工作区FME磨破ld Tour 2016. Each worker FME Engine is started using a long-ago added mode (as part of early FME Server experimentation) where it creates a TCP/IP channel and then waits for a driver application to connect and send it work. In this example we create up to 10 engines.

工人脚本

Here is the script (start_containers_noweb.sh）创建运行FME引擎等待在端口7777上工作的服务。

＃！/ bin / bash scriptpath =“$（cd”$（dirname“$ {bash_source [0]}”）“&& pwd）”Docker Swarm Init.docker service create--name fme_engine -p 7777：7777  - 汇位类型=绑定，目标= / fmeengine /工作空间，源= $ scriptpath / workspaces --mount类型=绑定，目标= / fmeengine /结果，source = $ scrippath /结果 --mount类型=绑定，目标= / fmeengine / logs，source = $ scrippath / logs s亚搏在线afesoftware / fme-engine-2017Docker运行- 它-d -p 5000：8080 -v /var/run/docker.sock:/var/run/docker.sock manomarks /可视化

在脚本中，您可以看到3 docker命令：

Docker Swarm Init.

This command initializes a swarm and puts the current node into the swarm.

docker service create

This command creates a single service for the fme_engine. It specifies that the engine port of 7777 is mapped to the same port on the host. The engine is also told where it can find its workspaces to run, the directory to put its logfiles, and where the results are to be placed.

Docker运行

最后一个命令只运行我们最喜欢的Docker服务可视化工具，该工具由Mano Marks. Always nice to visualize all the containers running.

FME磨破ker Workspace

Docker Magic在哪里？

The magic here is that as far as each worker is concerned, it is the only one. They all specify the same port number within the Docker container and Docker Swarm looks after all the details of multiplexing the requests between the workers and the driver FME.

推出10名工人

Step 1: Run the start_containers script.

FMEStandaloneEngine dcm$./start_containers_noweb.sh

Step 2: Check out the Visualization of the single running FME Engine Container (localhost:5000)

Mano标记显示1个容器运行FME的Visualizer。

Step 3: Spin up a total of 10 engines

现在告诉群体我们想要有10个FME发动机运行。

FMEStandaloneEngine dcm$Docker服务规模FME_ENGINE = 10

It is as simple as that. We now have 10 FME Engines running.

第4步：验证它们是否再次运行（localhost：5000）

Mano标记显示4个容器的Visualizer运行FME。

运行驱动程序工作区

Now that the swarm is launched with all its FME Engines we are now ready to run the driver workspace.

To do this, simply run the driver workspace above in FME Workbench. This is run outside of the Docker Swarm. If you don’t have FME then you canTry FME for free.

Verifying the Results

To see the Air Quality reports being generated by the FME Workspace, simply go into the “results” directory and you will find web pages of the Air Quality requested, one web page for each requested City. Example:

为什么这只是一个玩具演示

演示显示Docker Swarm如何摘要网络，以便在形成群体的不同容器和服务器上共享单个TCP / IP端口。DEMO工作相同，所有容器是在单个机器上还是在许多机器上传播。

This isn’t production quality as it is easy to overrun the TCP/IP channels in a heavily loaded system. There is no queuing going on at all. Swarm is simply sending the worker messages to the different worker containers waiting on the read side of the socket.

Try It Yourself – It’s Fun!

如果您想自己尝试，您会发现您需要的一切。谢谢将所有这一切放在一起，赠送！

• Docker集线器链接-https://hub.docker.com/r/safesoftware/fme-engine-2017/
• github存储库 -https://github.com/亚搏在线safesoftware/fme-standalone-engine.

观看视频也是。

下一步

在下一个博客中，我们将采取我们在此学到的内容，以建立一个真正的FME服务器的弹性可扩展原型。我们将利用Docker，云技术和FME服务器。

几乎太多乐趣。

Don Murray

唐是安全软件的联合创始人和总统。亚搏在线亚搏在线安全软件最初是在与林业行业共享空间数据的项目中为BC政府做的工作。在该项目期间，Don等联合创始人Dale Lutz实现了像FME这样的数据集成平台。当Don没有引诱他有多喜欢XML时，你可以找到唐在安全的时候，安全地将FME产品带到一个下一级。亚搏在线您还将在道路上找到他与客户和合作伙伴谈话，以了解他们希望看到的新FME功能的更多信息。

格兰特阿罗德

Grant is a DevOps Technical Lead for FME Server and works on all types of deployments for FME Server. In his spare time, Grant likes to play games and plays guitar.