Accesses the Amazon Simple Storage Service (S3) file storage service to upload, download, or delete files and folders or list file/folder information from an S3 account.
Typical Uses
- Manage datasets on S3 by uploading, downloading, and deleting files and folders
- Transfer a file's contents (such as XML or raster) into or out of an attribute in FME
- Read downloaded S3 data using theFeatureReader, or upload data written by theFeatureWriterto S3
- Retrieve file and folder names, paths, links and other information from S3 to use elsewhere in a workspace.
它是如何工作的？
S3Connector使用您的Amazon S3帐户凭据（通过先前定义的FME Web连接，或者通过从变压器设置新的FME Web连接）来访问文件存储服务。
Depending on your choice of actions, it will upload or download files, folders, and attributes; list information from the service; or delete items from the service. On uploads, link attributes can be added to the output features. On List actions, file/folder information are added as attributes.
例子
例子：Downloading a dataset
In this example, the S3Connector is used to download an Esri Geodatabase from S3. After creating a valid web connection to an S3 account (which can be done right in the Account parameter), and browsing to the geodatabase folder, the Bucket and Path to the object are retrieved, and a destination for the download is selected.
AFeatureReaderis added to read the newly downloaded dataset. Here, the PostalAddress feature type will be further processed elsewhere in the workspace.
通过在工作区中执行下载，每次运行工作空间时都会刷新地理数据库。
例子：Downloading and using a raster as an attribute
In this example portion of a workspace, the S3Connector is used to download a raster orthoimage from S3 into an attribute.
The file is read from S3, and the contents stored as a blob attribute. Then aRasterReplaceris used to interpret the blob into a usable raster format.
The combination of these two transformers avoids having to download the image to local storage and re-read it. A similar technique can be used for point cloud files, using thePointCloudReplacer变压器。
使用说明
- 此变换器不能用于直接移动或复制不同S3位置之间的文件。但是，多个S3Connectors可用于使用中间下载移动或复制文件。
- TheFeatureReadercan access S3 directly, without using the S3Connector. In this case, a local copy of the dataset will not be created.
Configuration
Input Ports
Input
此变形器接受任何功能。
输出端口
Output
该变压器的输出将根据所执行的S3动作而变化。
- After an上传action, URLs to the file may be saved to the Direct Link attribute, and the bucket and path to the object will saved to the Full Path attribute.
- ADownloadaction will output a new feature and can save to either a file, folder or attribute.
- A删除操作将输出一个新功能。
- AListaction will output a new feature for each file/folder found in the path specified. Each of these new features will have attributes listing various pieces of information about the object.
Features that cause the operation to fail are output through this port. Anfme_reimpe_code.attribute, having the valueERROR_DURING_PROCESSING，将添加，以及更具描述性的fme_rejection_message属性包含更具体的细节，以及失败的原因。
Note: ">Note:如果a feature comes in to the S3Connector already having a value forfme_reimpe_code., this value will be removed.
Rejected Feature Handling:可以设置为终止翻译或在遇到被拒绝的功能时继续运行。此设置可用作默认设置FME option和A.工作区参数.
Parameters
验证
| Credential Source | The S3Connector can use credentials from different sources. Using a web connection integrates best with FME, but in some cases, you may wish to use one of the other sources.
|
| Account | 凭据源是可用的网页Connection. To create an S3 connection, click the 'Account' drop-down box and select 'Add Web Connection...'. The connection can then be managed via Tools -> FME Options... -> Web Connections. |
| Region | The AWS Region in which the specified桶resides. If the default value,US East, N. Virginia (us-east-1), is specified, and the specified Bucket does not reside there, the operation will still succeed. However, to optimize latency, it is best practice to specify the correct region. |
| Alternate Endpoint | 如果指定，请使用备用S3兼容的API。这些通常是来自AWS以外的提供商或本地解决方案的对象存储服务。 |
| Verify SSL Certificates |
|
| Requester Pays | 一些公共桶要求请求者支付执行的运营费用。将此参数设置为Yeswill allow AWS to charge the account associated with the provided credentials to be charged. |
| Access Key ID and Secret Access Key | 凭据源是可用的Embedded. An access key ID and secret access key can be specified directly in the transformer instead of in a web connection. |
| Session Token | 选修的。凭据源是可用的Embedded. If specified, use the temporary security credentials to connect to S3. |
Request
| Action | 执行的操作类型。选择是：
|
The remaining parameters available depend on the value of the Request > Action parameter. Parameters for each Action are detailed below.
Parameters -删除Action
删除Options
桶 |
The bucket the file or folder is in. If valid credentials have been provided as a web connection, you may browse for a bucket. |
Path |
The full path of a file or folder on S3 to delete. If valid credentials have been provided as a web connection, you may browse for a location. |
Parameters -Download Action
Download Options
桶 |
The bucket the file or folder is in. If valid credentials have been provided as a web connection, you may browse for a bucket. |
Path |
The full path of a file or folder on S3 to download. If valid credentials have been provided as a web connection, you may browse for a location. |
S3加速 |
如果Yes，允许Amazon S3在桶上传输加速，如果启用则。要启用加速，请参阅http://docs.aws.amazon.com/AmazonS3/latest/UG/enable-bucket-transfer-acceleration.html. 允许S3加速时，请记住以下内容：
|
Destination
| Download as | Select whether to store the downloaded data in a File, Folder or Attribute.
|
| 下载到属性 | Specify the attribute that will store the contents of the downloaded file. Valid for Download as Attribute only. |
| 响应身体编码 | 如需下载作为属性，可以指定字符集编码以确保输出处于人类可读的形式。 如果Auto Detect from HTTP Headeris specified but there is no encoding in the Content-Type metadata associated with the data, the output will be returned as raw bytes. |
| Download to Folder | 指定将存储下载文件的文件夹的路径。有效地下载为文件或文件夹。 |
| 如果File Exists | Specify what to do if the file already exists locally. Valid for Download as File or Folder.
|
| Include Subfolders | Choose whether to download subfolders of the S3 Source or not. Valid for Download as Folder only. Default:No. |
Parameters -List Action
List Options
桶 |
该桶中的桶。如果已作为Web连接提供有效凭据，则可以浏览桶。 |
Path |
The full path to the folder on S3 to list. If valid credentials have been provided as a web connection, you may browse for a location. |
Include Subfolders |
如果设置为Yes, then subfolders are recursively traversed, and their contents are included in the result. Default:No. |
输出属性
提供默认属性名称，可能会被覆盖。
File or Folder Name |
Specify the attribute to hold the name of an object on S3. |
桶Name |
Specify the attribute to hold the ID of an object on S3. |
Full Path |
Specify the attribute to hold the full path to an object on S3. |
URL. |
Specify the attribute to hold the URL of an object on S3. |
文件大小 |
Specify the attribute to hold the size of a file object on S3. |
Last Modified |
Specify the attribute to hold the last modified date of an object on S3. |
File or Folder Flag |
指定属性以在S3上保留对象的类型（文件或文件夹）。 |
Relative Path |
Specify the attribute to hold the relative path to an object on S3. |
Parameters -上传Action
Data Source
上传 |
The type of data to be uploaded.
When working with large objects, File is a better choice than Attribute as the data will be streamed directly from disk and not require that the object be stored entirely in memory on a feature. S3在特定文件夹中处理相同名称的文件上传，作为重复项，但不允许在特定文件夹中的多个实例。您必须将文件上传到不同的文件夹，而不是现有的文件夹，或者原始将被覆盖。 When wanting to upload content from a folder, it is better to upload as a folder instead of setting a fixed path and sending multiple features into the connector to upload as a file. |
文件上传 |
The file to be uploaded to S3 if上传is set toFile. |
文件夹to Upload |
The folder to be uploaded to S3 if上传is set to文件夹. |
Include Subfolders |
Choose whether to upload subfolders of the文件夹to Upload与否。 |
Attribute to Upload as File |
从属性提供的要上载的数据上传is set toAttribute. |
上传Options
| 桶 | 该桶中的桶。如果已作为Web连接提供有效凭据，则可以浏览桶。 |
| Path | The full path to the folder on S3 to upload to. If valid credentials have been provided as a web connection, you may browse for a location. |
| 上传with File Name | The name of the file created from the data supplied inAttribute to Upload as File. The name must include a filename extension (for example, .txt, .jpg, .doc). |
| 如果File Exists | Specify what to do if the file already exists on S3.
|
S3加速 |
如果Yes，允许Amazon S3在桶上传输加速，如果启用则。要启用加速，请参阅http://docs.aws.amazon.com/AmazonS3/latest/UG/enable-bucket-transfer-acceleration.html. 允许S3加速时，请记住以下内容：
|
| Encryption |
|
| Permissions | The predefined set of grantees and permissions to store with each uploaded object. For more information, seehttp://docs.aws.amazon.com/amazons3/latest/dev/acloverview.html#cannedacl.. |
| Generate Presigned URL | 如果enabled, a presigned URL is generated for the uploaded file, and stored in the_PRESIGNED_URL.attribute. This allows access to the file without credentials for the bucket. For more information, seehttps://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html. |
| Expiry | When generating a presigned URL, optionally specify a number of seconds for which the link will remain valid. |
| Metadata | You can specify header fields in the HTTP upload request. UnderName, specify a header field. You can select from a drop-down list of predefined fields that include metadata and ACL permissions parameters. Alternatively, enter fields manually. UnderValue, specify a field value. Custom metadata fields will automatically be prefixed with ‘x-amz-meta-’ by AWS. |
输出属性
| File or Folder Name | Specify the output attribute that will store the name of the file or folder. For example, the attribute could contain ‘CityParks.gml’. |
| Full Path | Specify the output attribute that will store the full path of the file/folder that was just uploaded. For example, the attribute could contain ‘city_data/GML/CityParks.gml’. |
桶Name |
Specify the output attribute that will store the name of the bucket to which the file was uploaded . |
URL. |
Specify the output attribute that will store the URL to the file or folder on S3. |
编辑Transformer Parameters
Using a set of menu options, transformer parameters can be assigned by referencing other elements in the workspace. More advanced functions, such as an advanced editor and an arithmetic editor, are also available in some transformers. To access a menu of these options, click
beside the applicable parameter. For more information, seeTransformer Parameter Menu Options.
Defining Values
There are several ways to define a value for use in a Transformer. The simplest is to simply type in a value or string, which can include functions of various types such as attribute references, math and string functions, and workspace parameters. There are a number of tools and shortcuts that can assist in constructing values, generally available from the drop-down context menu adjacent to the value field.
如何设置标准ameter Values
Using the Text Editor
The Text Editor provides a convenient way to construct text strings (including regular expressions) from various data sources, such as attributes, parameters, and constants, where the result is used directly inside a parameter.
Using the Arithmetic Editor
算术编辑器提供了一种方便的方法来构建来自各种数据源的数学表达式，例如属性，参数和特征函数，其中结果直接在参数内使用。
Conditional Values
Set values depending on one or more test conditions that either pass or fail.
Parameter Condition Definition Dialog
Content
表达式和字符串可以包括许多函数，字符，参数等。
When setting values - whether entered directly in a parameter or constructed using one of the editors - strings and expressions containing String, Math, Date/Time or FME Feature Functions will have those functions evaluated. Therefore, the names of these functions (in the form @<function_name>) should not be used as literal string values.
Content Types
| These functions manipulate and format strings. | |
Special Characters |
文本编辑器中提供了一组控制字符。 |
| Math functions are available in both editors. | |
| Date/Time Functions | 文本编辑器中提供日期和时间函数。 |
| 这些运算符可在算术编辑器中使用。 | |
| These return primarily feature-specific values. | |
| FME and workspace-specific parameters may be used. | |
| Creating and Modifying User Parameters | Create your own editable parameters. |
Dialog Options - Tables
Transformers with table-style parameters have additional tools for populating and manipulating values.
Table Tools
行重新排序
|
启用后，单击行项目后。选择包括：
|
Cut, Copy, and Paste
|
启用后，单击行项目后。选择包括：
Cut, copy, and paste may be used within a transformer, or between transformers. |
Filter
|
Start typing a string, and the matrix will only display rows matching those characters. Searches all columns. This only affects the display of attributes within the transformer - it does not alter which attributes are output. |
Import
|
导入使用从数据集读取的一组新属性填充表。特定应用在变压器之间变化。 |
Reset/Refresh
|
Generally resets the table to its initial state, and may provide additional options to remove invalid entries. Behavior varies between transformers. |
Note:Not all tools are available in all transformers.
参考
Processing Behavior |
|
Feature Holding |
No |
| Dependencies | S3账户 |
| FME Licensing Level | FME Base Edition and above |
| Aliases | S3Deleter, S3Downloader, S3ObjectLister, S3Uploader |
| History | Released FME 2019.0 |
FME Community
FME社区是演示亚搏国际在线官网，TOS，文章，常见问题和更多的地方。获取您的问题的答案，从其他用户学习，并建议，投票和评论新功能。
Search forall results about the S3Connectoron the FME Community.
例子may contain information licensed under the Open Government Licence – Vancouver and/or the Open Government Licence – Canada.