span8
span4
span8
span4
I currently have a custom Transformer that gets a parameter MaxTileCount (the value of _num_tiles) from a WebMapTiler and it counts the number of features and sends a HTTP message every 10 messages to show progress.
That is all working fine.However, now I have issue.The PNGwriter is no longer a PNGRaster writer, but has changed to a FeatureWriter that does a fanout.Well that takes a lot of time, So I want to move the Notifier to later in the pipe, after the "pngWriter (FeatureWriter). The issue I have is that I don't know how to count anymore the number of files written by the FeatureWriter as they are emitted, nor do I have access to _num_tiles which is the total number of tiles coming from the WebMapTiler.
In simple words, I want to send a HTTPPost message withcount/totalas the files are written by the FeatureWriter any idea who to do this?
The second notifier fails.So I disabled it.
The FeatureWriter itself doesn't have your desired functionality (progress monitoring), but if the major issue was that Dataset Fanout takes a long time, possibly this approach could reduce the time.
Well...that's a very interesting requirement.Since the FeatureWriter is not outputting the summary as one feature at a time, we have to figure out another way.I can think of a few...
1) Put the writing (FeatureWriter or a plain writer) inside a second workspace and use the WorkspaceRunner to run it.That will return a feature as soon as the job is completed, and you can do your count to ten on those.
2) Somehow scan for files as they are written so you have that live count.Perhaps each feature is also sent to a custom transformer that loops around - waiting for a files existence - before the feature is output and counted.Another way would be to perhaps pass the feature into a Python script that checks for the file and loops until it exists.
To keep things simpler, these too could go into a separate workspace.It could store the count in a text file readable by the master, or just pass features back (maybe try a Sender/Receiver pair?)
3) The simplest - though least accurate - solution, might be to just put a decelerator transformer into the workspace.Say you know each tile takes about 30 minutes to write (for example) just put a 30 minute delay in before the Counter.It won't be as accurate, but it will give you an approximate idea of how the process is going.Or the delay could be related to an attribute - eg if you have an idea of file size or number of rows/columns, the delay could be proportionate to that, making it more accurate.
在前两个的情况下,我不太确定这s might interact with your fanout, or how the features relate to the files, but I think both are capable of being implemented without too much of a problem.
I hope one of these helps.
Mark
Hi@ygutfreund, the FeatureWriter outputs summary features after completion of writing and each of them has an attribute called "_total_features_count", which stores the number of features (PNG files in this case) that have been written into a destination dataset (folder in this case).I think you can get the total number of written PNG files from this attribute and then send a message using an HTTPCaller.
You can use the VariableSetter to create a variable from the _numtiles attribute from the WebmapTiler.After the FeatureWriter you can retrieve the value from the variable using the VariableRetriever transformer and use that for your count.
PNG Optimization2 Answers
PNG-writer prints out extra PNG-files1 Answer
How to write Raster-Files with a World-File using the FeatureWriter with Generic-Format2 Answers
Has anyone used FME to create a map?6 Answers
How to count FeatureWriter transformer's output into "Feature Written Summary" of the log?3 Answers
© 2020 Safe Software Inc |Legal