一. Lease
- A Lease governs all the locks held by a single client.
* For each client there's a corresponding lease, whose
* timestamp is updated when the client periodically
* checks in. If the client dies and allows its lease to
* expire, all the corresponding locks can be released. - 翻译如下:Lease管理着一个client占用的所有锁。一个client对应着一个Lease,在client发送请求时,对应Lease的timestamp调整到最新时间。如果client死亡,让对应Lease过期,释放Lease管理的所有锁。
- 个人理解:Lease是一个文件写锁,当client需要写文件时,需要申请一个Lease。NameNode负责记录哪个文件上有Lease,Lease的客户是谁,超时时间。
- 成员变量
private final String holder; //客户端名 private long lastUpdate; //最近更新时间 private final Collection<String> paths = new TreeSet<String>(); //该客户端操作的文件集合
- 方法,其方法都很简单,随便举两个例子
/** Only LeaseManager object can create a lease */ private Lease(String holder) { this.holder = holder; renew(); } /** Only LeaseManager object can renew a lease */ private void renew() { this.lastUpdate = FSNamesystem.now(); } //replacing oldpath with newpath synchronized void changeLease(String src, String dst, String overwrite, String replaceBy) { } synchronized void removeLeaseWithPrefixPath(String prefix) { }
二. Monitor
- Monitor checks for leases that have expired, and disposes of them.
- 每两秒调用checkLeases()。checkLeases()检查每个Lease是否过期,如果过期,调用fsnamesystem.internalReleaseLeaseOne(oldest, path);
三. LeaseManager
- LeaseManager管理着所有的Lease。
- Lease和Monitor都是LeaseManager内部类。Lease的private构造函数保证额只有LeaseManager才能创建一个Lease。
- 成员变量
private final FSNamesystem fsnamesystem; //Lease.holder -> Lease private SortedMap<String, Lease> leases = new TreeMap<String, Lease>(); // Set of: Lease private SortedSet<Lease> sortedLeases = new TreeSet<Lease>(); //pathnames -> Lease private SortedMap<String, Lease> sortedLeasesByPath = new TreeMap<String, Lease>();
- 方法,无非就是对几个集合的增删改查操作,毫无难度。需要注意的是,addLease()并没有检查文件(src)上是否已经有Lease,这个需要由LeaseManager调用者保证。
synchronized Lease addLease(String holder, String src) { Lease lease = getLease(holder); if (lease == null) { lease = new Lease(holder); leases.put(holder, lease); sortedLeases.add(lease); } else { renewLease(lease); } sortedLeasesByPath.put(src, lease); lease.paths.add(src); return lease; } /** * Remove the specified lease and src. */ synchronized void removeLease(Lease lease, String src) { sortedLeasesByPath.remove(src); } /** * Reassign lease for file src to the new holder. */ synchronized Lease reassignLease(Lease lease, String src, String newHolder) { return addLease(newHolder, src); } /** * Remove the lease for the specified holder and src */ synchronized void removeLease(String holder, String src) { } /** * Finds the pathname for the specified pendingFile */ synchronized String findPath(INodeFileUnderConstruction pendingFile ) throws IOException { } /** * Renew the lease(s) held by the given client */ synchronized void renewLease(String holder) { }
四. LeaseManager类图
相关推荐
赠送jar包:hadoop-hdfs-client-2.9.1.jar 赠送原API文档:hadoop-hdfs-client-2.9.1-javadoc.jar 赠送源代码:hadoop-hdfs-client-2.9.1-sources.jar 包含翻译后的API文档:hadoop-hdfs-client-2.9.1-javadoc-...
赠送jar包:hadoop-hdfs-2.6.5.jar; 赠送原API文档:hadoop-hdfs-2.6.5-javadoc.jar; 赠送源代码:hadoop-hdfs-2.6.5-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-2.6.5.pom; 包含翻译后的API文档:hadoop...
赠送jar包:hadoop-hdfs-2.7.3.jar; 赠送原API文档:hadoop-hdfs-2.7.3-javadoc.jar; 赠送源代码:hadoop-hdfs-2.7.3-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-2.7.3.pom; 包含翻译后的API文档:hadoop...
赠送jar包:hadoop-hdfs-client-2.9.1.jar; 赠送原API文档:hadoop-hdfs-client-2.9.1-javadoc.jar; 赠送源代码:hadoop-hdfs-client-2.9.1-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-client-2.9.1.pom;...
赠送jar包:hadoop-hdfs-2.7.3.jar; 赠送原API文档:hadoop-hdfs-2.7.3-javadoc.jar; 赠送源代码:hadoop-hdfs-2.7.3-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-2.7.3.pom; 包含翻译后的API文档:hadoop...
赠送jar包:hadoop-hdfs-2.9.1.jar 赠送原API文档:hadoop-hdfs-2.9.1-javadoc.jar 赠送源代码:hadoop-hdfs-2.9.1-sources.jar 包含翻译后的API文档:hadoop-hdfs-2.9.1-javadoc-API文档-中文(简体)版.zip 对应...
赠送jar包:hadoop-hdfs-2.6.5.jar; 赠送原API文档:hadoop-hdfs-2.6.5-javadoc.jar; 赠送源代码:hadoop-hdfs-2.6.5-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-2.6.5.pom; 包含翻译后的API文档:hadoop...
赠送jar包:hadoop-hdfs-2.5.1.jar; 赠送原API文档:hadoop-hdfs-2.5.1-javadoc.jar; 赠送源代码:hadoop-hdfs-2.5.1-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-2.5.1.pom; 包含翻译后的API文档:hadoop...
赠送jar包:hadoop-hdfs-2.5.1.jar; 赠送原API文档:hadoop-hdfs-2.5.1-javadoc.jar; 赠送源代码:hadoop-hdfs-2.5.1-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-2.5.1.pom; 包含翻译后的API文档:hadoop...
赠送jar包:hadoop-hdfs-2.9.1.jar; 赠送原API文档:hadoop-hdfs-2.9.1-javadoc.jar; 赠送源代码:hadoop-hdfs-2.9.1-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-2.9.1.pom; 包含翻译后的API文档:hadoop...
hadoop-auth-3.1.1.jar hadoop-hdfs-3.1.1.jar hadoop-mapreduce-client-hs-3.1.1.jar hadoop-yarn-client-3.1.1.jar hadoop-client-api-3.1.1.jar hadoop-hdfs-client-3.1.1.jar hadoop-mapreduce-client-jobclient...
Hadoop 3.x(HDFS)----【HDFS 的 API 操作】---- 代码 Hadoop 3.x(HDFS)----【HDFS 的 API 操作】---- 代码 Hadoop 3.x(HDFS)----【HDFS 的 API 操作】---- 代码 Hadoop 3.x(HDFS)----【HDFS 的 API 操作】--...
hadoop-hdfs-2.4.1.jar
赠送jar包:hadoop-mapreduce-client-jobclient-2.6.5.jar; 赠送原API文档:hadoop-mapreduce-client-jobclient-2.6.5-javadoc.jar; 赠送源代码:hadoop-mapreduce-client-jobclient-2.6.5-sources.jar; 赠送...
hadoop-hdfs-2.7.3搭建flume1.7需要用到的包,还有几个包也有提供
赠送jar包:hadoop-yarn-api-2.5.1.jar; 赠送原API文档:hadoop-yarn-api-2.5.1-javadoc.jar; 赠送源代码:hadoop-yarn-api-2.5.1-sources.jar; 赠送Maven依赖信息文件:hadoop-yarn-api-2.5.1.pom; 包含翻译后...
Hadoop学习总结之四:Map-Reduce的过程解析
flume 想要将数据输出到hdfs,必须要有hadoop相关jar包。本资源是hadoop 2.7.7版本
二、实验环境 Windows 10 VMware Workstation Pro虚拟机 Hadoop环境 Jdk1.8 三、实验内容 1:hdfs常见命令: (1)查看帮助:hdfs dfs -help (2)查看当前目录信息:hdfs dfs -ls / (3)创建文件夹:hdfs dfs ...
hadoop-hdfs-2.2.0.jar 点击下载资源即表示您确认该资源不违反资源分享的使用条款