http://parand.com/say/index.php/2009/04/09/tokyo-cabinet-observations/
I’m using Tokyo Cabinet with Python tc for a decent sized amount of data (~19G in a single hash table) on OS X. A few observations and oddities:
* Writes slow down significantly as the database size grows. I’m writing 97 roughly equal sized batches to the tch table. The first batch takes ~40 seconds, and processing time seems to increase fairly linearly, with the last taking ~14 minutes. Not sure why this would be the case, but it’s discouraging. I’ll probably write a simple partitioning scheme to split the data into multiple databases and keep the size of each small, but it seems like this should be handled out of the box for me.
* [Update] I implemented a simple partitioning scheme, and sure enough it makes a big difference. Apparently keeping the file size small (where small is < 500G) is important. Surprising - why doens’t TC implement partitioning if it’s susceptible to performance issues with larger file sizes? Is this a python tc issue or a Tokyo Cabinet issue?
* [Also] Seems I can only open 53-54 tc.HDB()’s before I get an ‘mmap error’, limiting how much I can partition.
* Reading records that have already been read from the tch seems to go much faster on the second access (like an order of magnitude faster). I suspect this is the disk cache at work, but if anyone has extra info on this please enlighten me.
Another somewhat surprising aspect: using the tc library you’re essentially embedding Tokyo Cabinet in your app; I had assumed it was going to be network based access, but it’s not. You can do network access either using the memcached protocol or using pytyrant.
分享到:
相关推荐
尝试开发PHP的扩展,仅用于学习。目前仅能够在windows下编译通过。 Linux下的tokyo cabinet API与Windows与差别,写的时候是在windows下,准备在linux下编译时才发现tokyo cabinet API在Unix下与Windows下是不一样的
tokyo cabinet tyrant研究资料
Tokyo cabinet C 库的Lua绑定接口。 Tokyo cabinet 是一个管理数据库的库。该数据库是一个单一的数据文件,每个记录为关键字和值。每个关键字和值是可变长度的字节序。二进制数据和字符串都可作为关键字或值。每个...
Tokyo Cabinet Key-Value数据库及其扩展应用
Tokyo cabinet C 库代码的Java绑定接口。 Tokyo cabinet 是一个管理数据库的库。该数据库是一个单一的数据文件,每个记录为关键字和值。每个关键字和值是可变长度的字节序。二进制数据和字符串都可作为关键字或值。...
Tokyo-Cabinet.tar.gz
东京内阁是QDBM的后继者,QDBM是与DBM系列类似的高性能数据库库。 它还支持哈希和B树数据库,不需要任何服务器进程。 与QDBM相比,整体速度有所提高。
东京橱柜食谱 使用 Chef 安装东京橱柜。 支持的平台 支持以下平台: ...在节点的run_list包含tokyo-cabinet : { " run_list " : [ " recipe[tokyo-cabinet::default] " ] } 作者 作者:坂锐( )
BNR持久性亚伦·希勒加斯(Aaron Hillegass) 2010年7月9日经过几年的抱怨,Core Data可能会变得更好,我认为我应该编写一个...安装首先,您需要下载Tokyo Cabinet: : (有一个sourceforge页面,但是最新的版本似乎
哪吒(Nezha)是中国神话故事里的少年战神,我们以其作为基础Tokyo Cabinet的简单分布式KV存储系统原型项目的代号。 它包含configdb lib(configdb.h / libconfigdb.so)和一个命令行测试程序(Nezha) 执行make ...
东京暴君数据库的客户端封装,使东京暴君可以通过网络连接
Tokyo cabinet C库的Perl绑定代码API。 Tokyo cabinet 是一个管理数据库的库。该数据库是一个单一的数据文件,每个记录为关键字和值。每个关键字和值是可变长度的字节序。二进制数据和字符串都可作为关键字或值。每...
Microlog Cabinet Manager 2003 is a utility for opening and creating Microsoft CAB compressed files. CAB is a file compression format used by Microsoft to distribute many of their products, including ...
罗马食谱使用 Chef 安装 ROMA。 关于启动 ROMA,请阅读 ROMA 网站。 支持的平台支持以下平台: centos 乌本图属性钥匙类型描述默认['罗马']['gem_path'] 细绳选择您使用的宝石/选择/rbenv/垫片/宝石['罗马']['target...
Tokyo cabinet C 库的 Ruby绑定代码API Tokyo cabinet 是一个管理数据库的库。该数据库是一个单一的数据文件,每个记录为关键字和值。每个关键字和值是可变长度的字节序。二进制数据和字符串都可作为关键字或值。每...
Tokyo Cabinet 是一个DBM的实现。这里的数据库由一系列key-value对的记录构成。key和value都可以是任意长度的字节序列,既可以是二进制也可以是字符串。这里没有数据类型和数据表的概念。 当做为Hash表数据库使用时,...
Laravel开发-cabinet Laravel 4文件上传包。
用于发布ActiveX的CabinetSDK
Cabinet Vision (CV) 板式家具拆单软件 V2021 571中文语言包
Microsoft Cabinet Templatessource.zip