HFTP Guide

Introduction（说明）

HFTP is a Hadoop filesystem implementation that lets you read data from a remote Hadoop HDFS cluster. The reads are done via HTTP, and data is sourced from DataNodes. HFTP is a read-only filesystem, and will throw exceptions if you try to use it to write data or modify the filesystem state.

HFTP是使hadoop文件系统从远程hdfs集群读取数据的一种实现，读取时通过http协议完成的，并且数据源来自于datanodes。HFTP时一种只读文件系统，并且会抛出异常如果你尝试通过他去写数据或者修改文件系统状态。

HFTP is primarily useful if you have multiple HDFS clusters with different versions and you need to move data from one to another. HFTP is wire-compatible even between different versions of HDFS. For example, you can do things like: hadoop distcp -i hftp://sourceFS:50070/src hdfs://destFS:8020/dest. Note that HFTP is read-only so the destination must be an HDFS filesystem. (Also, in this example, the distcp should be run using the configuraton of the new filesystem.)

HFTP主要被用在如果你有多个不同版本的HDFS集群，并且你需要从一个集群移动数据到另一个集群时。HFTP时wire-compatible甚至在两个不同的HDFS版本之间。例如，你可以像这样做一些事：hadoop distcp -i hftp://sourceFS:50070/src hdfs://destFS:8020/dest.注意HFTP是只读的并且目标端必须是一个HDFS文件系统。（因此，在这个例子中，dictcp应该被运行在使用了新文件系统配置的集权中。）

An extension, HSFTP, uses HTTPS by default. This means that data will be encrypted in transit.

一个扩展，FSFTP，使用https协议，这意味着数据在传输过程中被加密的。

Implementation（实现）

The code for HFTP lives in the Java class org.apache.hadoop.hdfs.HftpFileSystem. Likewise, HSFTP is implemented in org.apache.hadoop.hdfs.HsftpFileSystem.

HFTP的代码编写在java类org.apache.hadoop.hdfs.HftpFileSystem.HSFTP的实现类是org.apache.hadoop.hdfs.HsftpFileSystem.

Configuration Options

Name	Description
dfs.hftp.https.port	the HTTPS port on the remote cluster. If not set, HFTP will fall back on dfs.https.port.
hdfs.service.host_ip:port	Specifies the service name (for the security subsystem) associated with the HFTP filesystem running at ip:port.

巴特西

HFTP Guide

最新文章

热门文章