Mon, 07/09/2007 - 16:57 — Todd Hoff
A fascinating and detailed story of how LiveJournal
evolved their system to scale. LiveJournal was an early player in the
free blog service race and faced issues from quickly adding a large
number of users. Blog posts come fast and furious which causes a lot of
writes and writes are particularly hard to scale. Understanding how
LiveJournal faced their scaling problems will help any aspiring website
builder.
Site: http://www.livejournal.com/
Information Sources
LiveJournal
- Behind The Scenes Scaling Storytime
Google Video
Tokyo Video
2005 version
Platform
Linux
MySql
Perl
Memcached
MogileFS
Apache
What's Inside?
Scaling from 1, 2, and 4 hosts to cluster of servers.
Avoid single points of failure.
Using MySQL
replication only takes you so far.
Becoming IO bound kills scaling.
Spread out writes and reads for more parallelism.
You can't keep adding read slaves and scale.
Shard
storage approach, using DRBD
, for maximal throughput. Allocate shards based on roles.
Caching
to improve performance with memcached. Two-level hashing to distributed RAM.
Perlbal
for web load balancing.
MogileFS, a distributed file system, for parallelism.
TheSchwartz
and Gearman for distributed job queuing to do more work in parallel.
Solving persistent connection problems.
Lessons Learned
Don't be afraid to write your own software to solve
your own problems. LiveJournal as provided incredible value to the
community through their efforts.
Sites can evolve from small 1, 2 machine setups to
larger systems as they learn about their users and what their system
really needs to do.
Parallelization is key to scaling. Remove choke
points by caching, load balancing, sharding, clustering file systems,
and making use of more disk spindles.
Replication has a cost. You can't just keep adding more and more read slaves and expect to scale.
Low level issues like which OS event notification
mechanism to use, file system and disk interactions, threading and even
models, and connection types, matter at scale.
Large sites eventually turn to a distributed queuing and scheduling mechanism to distribute large work loads across a grid.
分享到:
相关推荐
LiveJournal架构现状概况,网上收集的,但是来源忘记了!
从LiveJournal后台发展看大规模网站性能优化方法
LiveJournal LiveJournal数据库
Inside LiveJournal\'s Backend. or,. “holy hell that\'s a lot of hits!” April 2004. Brad Fitzpatrick. brad@danga....
LiveJournal 源存档 这是背后的代码库,从存储库的创建到官方存储库在 2014 年被LiveJournal.com, Inc关闭。 尽管官方存储库不再公开,但在此之前的源代码仍然根据 GNU GPL 获得许可,并由各种不同的版权所有者提供...
iPhoto的插件,可将图像上传到LiveJournal Scrapbook(Fotobilder)照片托管服务。
LiveJournal's Backend A history of scaling.PDF
LiveJournal的Java API:XML-RPC服务,http客户端,*。bml客户端。 需要Java 1.5+
LiveJournal API npm install livejournal 该软件包提供对LiveJournal的XMLRPC和JSONRPC API的访问。XMLRPC 例子// get posts inside the journal using XMLRPCLiveJournal . xmlrpc . getevents ( { journal : '...
简单的用户脚本使livejournal.com变得更好。 缩小太大的实时新闻主题。 Сужаетслишкомширокиетемыlivejournal.com。 支持语言:English
在Livejournal页面(国家、城市)上显示地理知识信息 Chrome扩展程序可在Livejournal页面上显示geo-ip信息Livejournal可以记录评论者的IP。 该工具基于记录的IP添加geo-ip信息。 您可以立即查看评论从世界的哪个地方...
博客 博客 LiveJournal 客户端
从LiveJournal后台发展看大规模网站性能优化方法 70 一、LiveJournal发展历程 70 二、LiveJournal架构现状概况 70 三、从LiveJournal发展中学习 71 1、一台服务器 71 2、两台服务器 72 3、四台服务器 73 4...
Pepys是完全用Java编写的LiveJournal的客户端。 它可以插入希望添加LiveJournal功能的任何Java程序中,并且可以在GPL下免费使用。
livejournal-ips Chrome 扩展程序可在 Livejournal 页面上显示地理 IP 信息 用法: 将此目录克隆到本地计算机; 在 Chrome 浏览器中,转到扩展页面并启用开发者模式(右上角的复选框); 点击“加载解压后的扩展...
Chrome扩展的地理IP信息,以显示有关LiveJournal页面的地理IP信息 LiveJournal可以选择签署IP的评论者。 此工具根据录制的IPS添加地理IP信息。 您可以立即查看评论发布的世界的哪个部分。 展示了国家,州省和城市。...
LiveJournal朋友监控器。 该应用程序是为需要在http://www.livejournal.com上跟踪他们的朋友列表的人们设计的。 wxWidgets框架似乎允许在Windows / MacOS主机以及带有GTK的* nix上执行此应用程序。
语言:English 简单的用户脚本,使livejournal.com成为一个更好的地方 缩小太大的实时新闻主题。 Сужаетслишкомширокиетемыlivejournal.com。