span8
span4
span8
span4
Tutorial: Getting Started with Automations
In this tutorial,you'll create an Automation that will trigger two parallel 亚搏在线workflows with different processing times,wait for both to complete,then proceed as one unified process.This type of 亚搏在线workflow may be of particular interest when,for example,updating several feature types in a database.Instead of one large job submitted to one FME engine,enjoy the performance gains of splitting your 亚搏在线workflow into separate jobs to be processed by many engines in parallel!Separating 亚搏在线workflows into several smaller jobs,rather than one large one,can also increase fault tolerance (if one job fails,other parts of the operation can continue).
With Automations,many workspaces may be chained together to run in series (and/or parallel) after a triggering event.Merge Actions are available that will cause an Automation to wait until all jobs upstream of the Merge point have completed,as long as they were triggered by the same event.This is particularly useful when an output requires many data processing steps,but not all of the jobs need to run in series.Provided more than one engine is available to FME Server,multiple jobs can be processed at once,increasing the speed of the automation.
In this tutorial,there are five workspaces;we will coordinate their running to produce a (fictional) final data product.The sample workspaces contain Decelerator transformers in order to demonstrate how an Automation would handle different job timing in a production 亚搏在线workflow.
Input files
AutomationsJobOrchestration-Begin.fsproject(built with FME Server 2019.0.0.1,b19246)
Finished Automation
AutomationsJobOrchestraion-Complete.fsproject(built with FME Server 2019.0.0.1,b19246)
From the downloads section above,download the linked FME Server projectAutomationsJobOrchestration-Begin.fsprojectand import it by clickingProjects>Projects,thenImport.
From the FME Server web interface,go toAutomations>Buildto create a new Automation.
On the Automations canvas,double-click the Trigger node.In this exercise,configure aSchedule InitiatedTrigger on a daily interval.Once the Automation is running,we'll be able to test it manually whenever we want.
Configure the next Action as aRun Workspace.The actual function (or lack thereof!) of these workspaces isn't important--imagine that this is the beginning of a nightly data update into a database.Look in the repository created when importing the server project (Automations Exercises),and addSpeedyDataUpdate.fmw.There are no published parameters to configure for this workspace.
If the node tray is not already expanded,click thesymbol and drag anotherAction onto the canvas downstream of the Trigger.Click the output port of the Trigger and drag a connection to the input port of the new Action.Configure this new Action as a Run Workspace.This time,addLongerDataUpdate.fmw.There are no published parameters to configure for this workspace.
Drag out a third Action node and configure it as aRun Workspacedownstream of the success port ofLongerDataUpdate.fmw.RunPostProcessing_LDU.fmwhere.
Downstream of all three Run Workspace Actions,place aMergeAction.Connect it to the success ports of both branches of your Automation.
Downstream of the Merge,configure aRun WorkspaceAction to runValidation.fmw.Now that all the planned updates to the database are prepared,it is time for a validation routine before the changes are reconciled and posted.Downstream of the success port of the Action that runs the data validation workspace,configure aRun Workspaceto runMakeDataProduct.fmw.Imagine that you're now set up to generate a nightly report that will be ready in the morning with the previous day's work incorporated!
Downstream of the failure port of theValidation.fmwAction,configure aSend an EmailExternal Action (or another external notification of your choice) to alert you if the night's data upload has failed validation.
As in the articleRun a Workspace in Response to Incoming Email,useLoad Templateor manually enter your email server information.If you are using an SMTP server that requires authentication (likely with popular email providers),you'll need to enter values in theSMTP Account (optional)andPassword (optional)fields.Input anEmail Toaddress you can check,and add anEmail From(the same as your account email address) address.
In theEmail Attachmentfield,use the down arrow to selectWorkspace>Job Log.
In theEmail Bodyfield,selectGeneral>Event as JSON,or chooseText Editorto compose a friendly message that you can automatically populate with details using JSON keys from the Automation.
To get an email with the job log of any other failed jobs,connect the failure ports of those jobs to the email Action as well (or add another Email Action).Since the workspaces included in this exercise will not fail jobs,test that emails will be sent by also connecting the emailer to the success port of your validation job.
To preserve your progress,clickMenuabove the Automations canvas,thenSave As.Choose a name and add some descriptive tags (optional),then clickOK.Finally,clickStart Automationin the upper right.
If you configured the trigger as a schedule and leftStart Immediatelychecked when configuring it,your Automation should complete its first run in short order.Wait a minute for the Jobs to finish,then checkMenu>View Triggered Jobs.You should see the five workspaces from your Automation listed,with the Automation's name asSource NameandSource TypeAutomations.To see the job log from any of the workspace jobs submitted by the Automation,you can click on its name here.
If you don't see a set of jobs from this Automation yet,manually trigger it.To do this,display the running Automation on the canvas (click its name in theAutomations>Managelist),then click into the Schedule node and clickTrigger.
The job logs will tell you what happened while a particular workspace was run,but for the big picture of what the Automation did when,return toAutomations>Manage,tick the checkbox for your parallel 亚搏在线workflows Automation,and selectView Log Filefrom theActionsmenu.
For future troubleshooting,you can search or filter the Automation log with the tools above the log entries.For now,click the clock button to show timestamps for each log entry.After the single-Action branch finishes its job,you should see a log entry,"(Automations) Received 1 of 2 notifications for merge action." From the time stamps,you'll see that the second chained job is not sent to an Engine to be processed until after the upstream job is complete,and that once the third job completes,there is another entry,"(Automations) Received 2 of 2 notifications for merge action.",after which any Actions after the Merge are processed.
Congratulations!You just built an Automation that can run jobs simultaneously without getting parts of the 亚搏在线workflow out of sync,and verified that it worked as intended.
Well done.Your Automation will now orchestrate two parallel 亚搏在线workflows that merge into one after all parallel jobs are complete.
Schedule a Workspace to Run with FME Server Automations
Create Self-Serve Access to Data with FME Server
Run a Workspace When Data Arrives in a Directory
Automating Dropbox 亚搏在线Workflows with FME Server
Making phone calls and sending automated SME messages using FME and Twilio
Automating Amazon S3 亚搏在线Workflows with FME Server
Introduction to Scheduling and Change Detection 亚搏在线Workflows in FME Server
© 2019 亚搏在线Safe Software Inc |Legal