span8
span4
span8
span4
This is a fantastic Christmas present.Thank you very much!
Where can we access the documentation on the API?
======================= FME 2017.0 "What's New" =======================
[Edit] OK.I found the doc in the FME 2017.0 beta build 17217 install directory :-)
fmeobjects.FMEBand / FMEBandProperties/ FMEBandTilePopulator / FMEPalette / FMERaster / FMERasterProperties / FMERasterTools / FMETile
Thank you Takashi for highlighting the new raster additions to the FME Objects Python API for FME 2017.0.Great sample applications!I just wanted to let everyone know that the updated 2017.0 Python API documentation,including the new Raster APIs,can now be accessed online athttps://docs.亚搏在线safe.com/fme/html/FME_Objects_Python_API/index.html
@takashi,you are correct.However,our raster dev team would like to clarify a few things:
ceil(number of raster rows / number of rows per tile) x ceil(number of raster columns / number of columns per tile)
Hi@DaveAt亚搏在线Safe,thanks for your response regarding the Number of Rows Per Tile.
OK.I understood that data contents of a raster band can consist of one or more tiles.The FMEBandTilePopulator.getTile() method will be called multiple times when populating the data contents to the band,if the number of rows/columnsper tileset to the band is less than the number of rows/columnsof the raster.The number of method callings is equal to the number of tiles.i.e.
ceil(number of raster rows / number of rows per tile) x ceil(number of raster columns / number of columns per tile)
and the starting row/column index in the given data matrix and the size will be passed via the arguments (startRow,startCol,tile) to the getTile() method for each tile creation.
Therefore,the MyUInt8BandTilePopulator.getTile() method in my exercise can also be defined like this using the arguments,and this definition allows any number of rows/columns per tile specified as the band properties.
class MyUInt8BandTilePopulator(fmeobjects.FMEBandTilePopulator): def __init__(self,dataArray): self.dataArray = dataArray def clone(self): return MyUInt8BandTilePopulator(self.dataArray) def getTile(self,startRow,startCol,tile): numRows,numCols = tile.getNumRows(),tile.getNumCols() endRow,endCol = startRow + numRows,startCol + numCols data = [self.dataArray[r][startCol:endCol] for r in range(startRow,endRow)] newTile = fmeobjects.FMEUInt8Tile(numRows,numCols) newTile.setData(data) return newTile
Is it correct?
An example to calculate statistics of cell values.
[Fixed: 2017-03-15]
# PythonCaller Script Example: Statistics of Raster Cell Values# Calculate fundamental statistics of cell values (except Nodata)# in the band(s) of the input raster.# This example only supports REAL64 and REAL32 but you can enhance# it easily if necessary.import fmeobjects,math class RasterBandStatisticsCalculator(object): def __init__(self): self.keys = [ 'interpretation','nodata','count','sum','min','max','range','median','mean','stdev','stdevp',] # Returns a tuple (tile object,interpretation name).# This method returns (None,'Unsupported') # when the specified interpretation was not supported,def tileAndInterpretation(self,interpretation,numRows,numCols): if interpretation == fmeobjects.FME_INTERPRETATION_REAL64: return (fmeobjects.FMEReal64Tile(numRows,numCols),'REAL64') elif interpretation == fmeobjects.FME_INTERPRETATION_REAL32: return (fmeobjects.FMEReal32Tile(numRows,numCols),'REAL32') ###################################################### # Add other interpretations here if necessary.###################################################### else: return (None,'Unsupported') # Returns a dictionary containing the statistics of specified band.def calculateStatistics(self,band,numRows,numCols): # Get the band properties.bandProperties = band.getProperties() interpretation = bandProperties.getInterpretation() # Create a tile that will be used to get cell values of the band.# The interpretation,number of rows,and number of columns # have to be consistent with the band properties.tile,interpret = self.tileAndInterpretation(interpretation,numRows,numCols) stats = {'interpretation': interpret} if tile != None: # Get all the cell values except Nodata as a list.values = [] nodataValue = band.getNodataValue() if nodataValue == None: for data in band.getTile(0,0,tile).getData(): values += data else: nodata = nodataValue.getData()[0][0] for data in band.getTile(0,0,tile).getData(): values += [v for v in data if v != nodata] stats['nodata'] = nodata # Calculate statistics.values.sort() count = len(values) stats['count'] = count if 0 < count: total = sum(values) stats['sum'] = total stats['min'] = values[0] stats['max'] = values[-1] stats['range'] = (values[-1] - values[0]) # Median m = count // 2 stats['median'] = (values[m] if count % 2 == 1 else (values[m-1] + values[m]) * 0.5) # Mean (average) avrg = float(total) / count stats['mean'] = avrg # Standard Deviation if 1 < count: s = sum([(v - avrg)**2 for v in values]) stats['stdev'] = math.sqrt(s / (count - 1)) stats['stdevp'] = math.sqrt(s / count) return stats def input(self,feature): raster = feature.getGeometry() if isinstance(raster,fmeobjects.FMERaster): rasterProperties = raster.getProperties() numRows = rasterProperties.getNumRows() numCols = rasterProperties.getNumCols() for i in range(raster.getNumBands()): stats = self.calculateStatistics(raster.getBand(i),numRows,numCols) for key in self.keys: attr = '_band{%d}.%s' % (i,key) if key in stats: feature.setAttribute(attr,stats[key]) else: feature.setAttributeNullWithType(attr,fmeobjects.FME_ATTR_REAL64) self.pyoutput(feature)
The second exercise: Calculate cell values,append a palette to the band,and try a raster tool.From theConway's Game of Life.
FME Challenge: The Game of Life
# PythonCaller Script Example: Conway's Game of Life# FME 2017.0 RC build 17254,2017-02-28# Each of the output features contains a raster representing live/dead cells.# The raster has a single band and the band has a palette.# Assuming that the input feature has these attributes.# 1.numGenerations (0 < integer): The number of generations to be processed.# 2.numRows (0 < integer): The number of rows in the resulting raster.# 3.numCols (0 < integer): The number of columns in in the resulting raster.# 4.initialRate (0 < real < 1): The rate of alive cells in the first generation.# A new attribute called '_generation' will be added to the output features,# which stores 1-based sequential number indicating the order of the generation.import fmeobjects,randomclass MyUInt8BandTilePopulator(fmeobjects.FMEBandTilePopulator): def __init__(self,dataArray): self.dataArray = dataArray def clone(self): return MyUInt8BandTilePopulator(self.dataArray) def getTile(self,startRow,startCol,tile): numRows,numCols = len(self.dataArray),len(self.dataArray[0]) newTile = fmeobjects.FMEUInt8Tile(numRows,numCols) newTile.setData(self.dataArray) return newTileclass ConwaysGameOfLife(object): def __init__(self): self.rasterTools = fmeobjects.FMERasterTools() def input(self,feature): # Get parameters from feature attributes.numGenerations = int(feature.getAttribute('numGenerations')) self.numRows = int(feature.getAttribute('numRows')) self.numCols = int(feature.getAttribute('numCols')) self.initialRate = float(feature.getAttribute('initialRate')) if 0 < numGenerations \ and 0 < self.numRows and 0 < self.numCols \ and 0.0 <= self.initialRate and self.initialRate <= 1.0: self.initialize() outNumRows,outNumCols = self.numRows * 10,self.numCols * 10 raster = None for i in range(numGenerations): raster = self.nextGeneration(raster) feature.setGeometry(self.rasterTools.resampleByRowCol( outNumRows,outNumCols,fmeobjects.FME_INTERPOLATION_NEARESTNEIGHBOR,raster)) feature.setAttribute('_generation',i + 1) self.pyoutput(feature) else: feature.setAttribute('_error','invalid parameter') self.pyputout(feature) def initialize(self): # Raster properties xSpacing,ySpacing = 1.0,1.0 xCellOrigin,yCellOrigin = 0.5,0.5 xOrigin,yOrigin = 0.0,0.0 xRotation,yRotation = 0.0,0.0 self.rasterProperties = fmeobjects.FMERasterProperties( self.numRows,self.numCols,xSpacing,ySpacing,xCellOrigin,yCellOrigin,xOrigin,yOrigin,xRotation,yRotation) # Band properties self.bandProperties = fmeobjects.FMEBandProperties( 'Game of Life',fmeobjects.FME_INTERPRETATION_UINT8,fmeobjects.FME_TILE_TYPE_FIXED,self.numRows,self.numCols) # Create a Palette object representing: # -------------------- # RGB24 # 0 220,220,220 # 1 128,0,0 # -------------------- key = fmeobjects.FMEUInt8Tile(1,2) key.setData([[0,1]]) value = fmeobjects.FMERGB24Tile(1,2) value.setData([[ 220,220,220,128,0,0,]]) self.palette = fmeobjects.FMEPalette('',key,value) # Return a raster representing the next generation,# or the first generation if the argument was None.def nextGeneration(self,prevRaster): nextData = [] if prevRaster == None: num = self.numRows * self.numCols alives = int(num * self.initialRate) seed = [1 for i in range(alives)] + [0 for i in range(num - alives)] random.shuffle(seed) for s in [i * self.numCols for i in range(self.numRows)]: nextData.append(seed[s:s + self.numCols]) else: # Get the data array of the previous generation.tile = fmeobjects.FMEUInt8Tile(self.numRows,self.numCols) prevData = prevRaster.getBand(0).getTile(0,0,tile).getData() # Create a temporary data array whose size is enlarged one row/column # per each edge - top,bottom,left,and right of the previous raster,# and set 0 to all the additional outer edge cells.# This data array makes it easy to compute the number of neighbor cells # alive in the previous generation for each cell.tmp = [[0 for i in range(self.numCols + 2)]] \ + [[0] + data + [0] for data in prevData] \ + [[0 for i in range(self.numCols + 2)]] # Create a new data array for the next generation.for i in range(self.numRows): row = [] for j in range(self.numCols): # Compute the number of neighbor cells alive in the previous # generation,and then determine if the cell can be alive # in the next generation according to the rules of the game.n = tmp[i+0][j] + tmp[i+0][j+1] + tmp[i+0][j+2] \ + tmp[i+1][j] + tmp[i+1][j+2] \ + tmp[i+2][j] + tmp[i+2][j+1] + tmp[i+2][j+2] if prevData[i][j] == 1: row.append(1 if n in [2,3] else 0) else: row.append(1 if n == 3 else 0) nextData.append(row) # Create and return a raster containing a single band.raster = fmeobjects.FMERaster(self.rasterProperties) band = fmeobjects.FMEBand(MyUInt8BandTilePopulator(nextData),self.rasterProperties,self.bandProperties) band.appendPalette(self.palette) # Add a palette to the band.raster.appendBand(band) return raster
I would share my first exercise to learn how to create a new raster geometry with the API.
# PythonCreator Script Example: Create a Feature containing a Raster# The raster will have a single band with UINT8 interpretation.import fmeobjects# Define a concrete class derived from FMEBandTilePopulator class.# An instance of this class will be used to create a tile# and populate it to a band (FMEBand instance).class MyUInt8BandTilePopulator(fmeobjects.FMEBandTilePopulator): def __init__(self,dataArray): self.dataArray = dataArray # Implement 'clone' method.# It will be called multiple times while creating a new band.def clone(self): return MyUInt8BandTilePopulator(self.dataArray) # Implement 'getTile' method.# You can create a new tile containing desired contents here.# It's not essential to use the parameters: startRow,startCol,tile.def getTile(self,startRow,startCol,tile): numRows,numCols = len(self.dataArray),len(self.dataArray[0]) newTile = fmeobjects.FMEUInt8Tile(numRows,numCols) newTile.setData(self.dataArray) return newTile # The following two methods won't be called while creating a new band.# It seems not to be essential to implement these methods in this case,# although the API doc says "This method must be implemented # in the FMEBandTilePopulator subclass".def setDeleteSourceOnDestroy(self,deleteFlag): pass def setOutputSize(self,rows,cols): return (rows,cols) class FeatureCreator(object): def __init__(self): pass def close(self): # Contents of a tile for a band to be created.# A list of row data,each element is a list of column values.dataArray = [ [ 0,128,0,128,0,128,0],[128,0,128,0,128,0,128],[ 0,128,0,128,0,128,0],[128,0,128,0,128,0,128],[ 0,128,0,128,0,128,0],] # Properties of a raster to be created.numRows,numCols = len(dataArray),len(dataArray[0]) # resolution xSpacing,ySpacing = 10.0,10.0 # cell spacing in ground units xCellOrigin,yCellOrigin = 0.5,0.5 # cell origin coordinates xOrigin,yOrigin = 0.0,numRows * ySpacing # left-top coordinates xRotation,yRotation = 0.0,0.0 # rotation angle in degrees # Create a new raster.rasterProperties = fmeobjects.FMERasterProperties(numRows,numCols,xSpacing,ySpacing,xCellOrigin,yCellOrigin,xOrigin,yOrigin,xRotation,yRotation) raster = fmeobjects.FMERaster(rasterProperties) # Create a new band and append it to the raster.# It's optional to specify Nodata value when creating a band.bandTilePopulator = MyUInt8BandTilePopulator(dataArray) bandName = 'My UInt8 Band' # can be set to empty.bandProperties = fmeobjects.FMEBandProperties(bandName,fmeobjects.FME_INTERPRETATION_UINT8,fmeobjects.FME_TILE_TYPE_FIXED,numRows,numCols) nodataValue = fmeobjects.FMEUInt8Tile(1,1) nodataValue.setData([[0]]) band = fmeobjects.FMEBand(bandTilePopulator,rasterProperties,bandProperties,nodataValue) raster.appendBand(band) # Create and output a feature containing the raster created above.feature = fmeobjects.FMEFeature() feature.setGeometry(raster) self.pyoutput(feature)
Thank you for sharing all your code here,Takashi,already helped me a lot!
I would appreciate help from someone for an error I get when trying to adapt this code to write out data to a Real64 band instead.To clarify: everything works fine if I just use the original code.
To simply try things out,I replaced all references to UInt8 in that code with Real64,threw the code into a PythonCreator and added an Inspector.
It runs fine until it calls the last line (as the print statement I added in shows)
# Create and output a feature containing the raster created above.feature = fmeobjects.FMEFeature()feature.setGeometry(raster)print("Message: All fine until here")self.pyoutput(feature)
The log reads:
PythonCreator_Creator(CreationFactory): Created 1 featuresMessage: All fine until hereStoring feature(s) to FME feature store file `C:\Users\bimta\AppData\Local\Temp\python_to_fme_214112\inspector.ffs'Stored 1 feature(s) to FME feature store file `C:\Users\bimta\AppData\Local\Temp\python_to_fme_214112\inspector.ffs'Saving spatial index into file 'C:\Users\bimta\AppData\Local\Temp\python_to_fme_214112\inspector.fsi'Finished saving spatial index into file 'C:\Users\bimta\AppData\Local\Temp\python_to_fme_214112\inspector.fsi'Inspector_Recorder(RecorderFactory): Failed to write feature data to `C:\Users\bimta\AppData\Local\Temp\python_to_fme_214112\inspector.ffs'Failed to write feature data to `C:\Users\bimta\AppData\Local\Temp\python_to_fme_214112\inspector.ffs'A fatal error has occurred.Check the logfile above for detailsPythonCreator(PythonFactory): PythonFactory failed to close properlyPythonCreator(PythonFactory): A fatal error has occurred.Check the logfile above for detailsSaving spatial index into file 'C:\Users\bimta\AppData\Local\Temp\python_to_fme_214112\inspector.fsi'Finished saving spatial index into file 'C:\Users\bimta\AppData\Local\Temp\python_to_fme_214112\inspector.fsi'A fatal error has occurred.Check the logfile above for detailsTranslation FAILED with 7 error(s) and 2 warning(s) (0 feature(s) output)FME Session Duration: 0.5 seconds.(CPU: 0.1s user,0.2s system)END - ProcessID: 21776,peak process memory usage: 48144 kB,current process memory usage: 48144 kBA fatal error has occurred.Check the logfile above for detailsProgram TerminatingTranslation FAILED.
A similar error turns up if I run without an inspector and feature cashing enabled.Then it simply points to another temporary folder.
I would greatly appreciate if someone here could help me out trying to understanding why it fails to write this feature.
Hi@bimtauer,it seems that the cell values should befloattype values when the band interpretation is Real64 or Real32.Try adding a decimal place to every constant value in the script,like this.
dataArray = [ [ 0.0,128.0,0.0,128.0,0.0,128.0,0.0], [128.0,0.0,128.0,0.0,128.0,0.0,128.0], [ 0.0,128.0,0.0,128.0,0.0,128.0,0.0], [128.0,0.0,128.0,0.0,128.0,0.0,128.0], [ 0.0,128.0,0.0,128.0,0.0,128.0,0.0], ]
nodataValue.setData([[0.0]])
So,I think the relevant description in the official documentation is wrong.In my observation,the data type in the description on FMEReal32/64Tile.getData and setData methods should belist[list[float]],rather thanlist[list[int]].
Hope someone from 亚搏在线Safe checks the links above.Who is in charge?@DaveAt亚搏在线Safe?
Hi@takashi,
I'm not in charge,but I am happy to create a problem report to have the docs for those methods updated (TECHPUBS-5786).I will notify you as soon as the doc is fixed.
Glad you found it.Merry Christmas!We're excited to open up raster manipulation to the Pythonistas out there.Bonus -- you can use either Python 3.x or 2.x as well!
© 2019 亚搏在线Safe Software Inc |Legal