span8
span4
span8
span4
Using Amazon SQS, or any queuing service is a great way of storing data that can't or doesn't need to get processed right away.It also provides some reassurance that a feature will only be deleted from the queue once it has been processed.A good overview of SQS in FME is availablehere.
Unlike the previous two methods discussed (HTTP POST to a Topic and WebSockets), using Amazon SQS doesn't require long term job processes or a lot of notifications (although it can do both!).
FME makes it easy to interact with SQS, either through the Notification Service (creating SQS Publication/Subscription) or using the SQS Transformers available.In this exercise the SQSReceiver transformer will be used to receive messages from an SQS queue.This transformer allows you to specify when the transformer will stop receiving features, either: Never, When a Queue is Empty or After a Fixed Number of Messages.This allows more flexibility with running jobs and engine use.
Every time a feature (row) is changed in a table in PostGIS, Node.js receives information about the feature.In the javascript file, when Node.js receives a feature from PostGIS it uses the aws-sdk module to push features into an Amazon SQS queue.
var pg = require ('pg');var request = require('request');var pgConString = "postgres://username:password@server:port/database";var AWS = require('aws-sdk');AWS.config.update({accessKeyId: 'accessKey', secretAccessKey: 'secretAccessKey'});var sqs = new AWS.SQS({region:'region'});console.log("Sending messages to SQS");pg.connect(pgConString, function(err, client) { if(err) { console.log(err);} client.on('notification', function(msg) { //console.log(msg);//console.log(JSON.parse(msg.payload));db_body = JSON.parse(msg.payload);var sqsmsg = { payload: 'a message' };var sqsParams = { MessageBody: JSON.stringify(db_body), QueueUrl: 'queueURL' };sqs.sendMessage(sqsParams, function(err, data) { if (err) { console.log('ERR', err);} console.log(data);});});var query = client.query("LISTEN watchers2");});
The complete workspace is available here:sqs-receiver-complete.fmw
See the Tutorial start page for more information on requirements.
Modify the postgis2SQS.js file to match your database and AWS SQS connection requirements.
In a command prompt window, navigate to the folder where the node executable is.In the command window type: node
如果什么都没有发生,这很好。任何错误的 code will get reported here.
In FME Workbench, add an SQSReceiver transformer and fill in the Amazon SQS connection details.Set the Stop Receiving parameter to 'When Queue is Empty'.
Now is the time to go and edit, create or delete some features from the PostGIS table.This can be using FME (in a separate workspace) or using a GIS.
After some features have been created, deleted or changed you should be able to see those changes in the AWS SQS Management Console:
Once there are features in the queue you can run the workspace.You should find the same number of features are received from SQS and output into the log message:
As the sqs_content attribute is JSON, a JSONFlattener would be helpful to turn each key into an attribute.The AttributeManager could then be used to create a message that could be used to notify someone of the changes to features.
Streaming features from PostGIS to FME using WebSockets
HTTP POST Requests to FME Server from Node.js
PostGIS to FME Server in Real Time (Node.js) Tutorial
FME Server as a WebHook for littleBits
Publish a littleBits workspace as an FME Server Workspace Subscription
Posting to the littleBits cloudBit
© 2020 Safe Software Inc |Legal