- 浏览: 2655224 次
- 性别:
- 来自: 北京
文章分类
最新评论
-
80后的童年2:
深入浅出MongoDB应用实战开发网盘地址:https://p ...
MongoDB入门教程 -
shliujing:
楼主在不是精通java和php的前提下,请不要妄下结论。
PHP、CakePHP哪凉快哪呆着去 -
安静听歌:
希望可以一给一点点注释
MySQL存储过程之代码块、条件控制、迭代 -
qq287767957:
PHP是全宇宙最强的语言!
PHP、CakePHP哪凉快哪呆着去 -
rryymmoK:
深入浅出MongoDB应用实战开发百度网盘下载:链接:http ...
MongoDB入门教程
Note3, Finding Bottlenecks: Benchmarking and Profiling
Benchmarking and profiling are two essential practices for finding bottlenecks
A benchmark measures your system's performance. This can help determine a system's capacity, show you which changes matter and which don't, or show how your application performs with different data
Profiling helps you find where your application spends the most time or consumes the most resources
Benchmarking answers the question "How well does this perform?" and profiling answers the question "Why does it perform the way it does?"
Why Benchmark?
Benchmarks can help you do:
Measure how your application currently performs
Validate your system's scalability
Plan for growth
Test your appplication's ability to tolerate a changing environment
Test different hardware, software, and operating system ocnfigurations
Benchmarking Strategies
There are two primary benchmarking strategies: you can benchmark the appplication as a whole, or isolation MySQL
These two strategies are known as full-stack and single-component benchmarking respectively
What to measure:
Transactions per time unit, Response time or latency, Scalability, Concurrency
Benchmarking Tactics
The first step in planning a benchmark is to identify the problem and the goal
Try to record as much additional information as you can during the benchmarks, such as CPU usage, disk I/O, and network traffic statistics, counters from SHOW GLOBAL STATUS
The best way to get accurate results is to design your benchmark to answer the question you want to answer
It's usually a good idea to automate the benchmark runs
Benchmarking Tools
Full-Stack Tools:
ab
ab is a well-known Apache HTTP server benchmarking tool, it shows how many requests per second your HTTP server is capable of serving
http_load
Similar tool in concept to ab, it's also designed to load a web server, but more flexible
JMeter
A Java application that can load another application and measure its performance, much more complex
Single-Component Tools:
mysqlslap
mysqlslap simulates load on the server and reports timing information
sysbench
sysbench is a multithreaded system benchmarking tool
You can measure the performance of file I/O, the OS scheduler, memory allocation and transfer speed, POSIX threds, and the database server itself
sysben supports scription in the Lua language, which makes it very flexible for testing a variety of scenarios
Database Test Suite
a test kit for running benchmarks similar to some industry-standard benchmarks, such as those published by TPC
MySQL Benchmark Suite
MySQL's own benchmark suite, single-threaded and measure how quickly the server executes queries
Super Smack
a benchmarking, stress-testing, and load-generating tool for MySQL and PostgreSQL
It's a complex, powerful tool that lets you simulate multiple users, load test data into the database, and populate tables with randomly generated data
Benchmarking Examples
Profiling
Database access if often the bottleneck in application
Bottlenecks can also be caused by any of the following:
External resources, such as calls to web services or search engines
Operations that require processing large amounts of data in the application, such as parsing big XML files
Expensive operations in tight loops, such as abusing regular expressiosn
Badly optimized algorithms, such as naive search algorithms to find items in lists
Application profiling can help you find the bottlenecks, and it's an important step in monitoring and improving overall performance
We recommend that you include profiling code in every new project you start
Your profiling code should gather and log at least the following:
Total execution time, or "wall-clock time"(in web applications, this is the total page render time)
Each query executed, and its execution time
Each connection opened to the MySQL server
Every call to an external resource, such as web services, memcached, and externally invoked scripts
Potentially expensive function calls, such as XML parsing
User and system CPU time
MySQL has two kinds of query logs: the general log and the slow log
The general log writes out every query as the server receives it, even some events such as connecting and disconnecting and queries may not even be executed due to errors
The slow log contains only queries that have executed
In particlular, slow log logs queries that take more than a specified amount of time to execute
Log analysis tools:
mysqldumpslow
mysql_slow_log_filter
mysql_slow_log_parser
mysqlsla
SHOW STATUS/SHOW PROFILE
Operating System Profiling
netstat
vmstat
iostat
mpstat
strace
OProfile
pgrof
Intel VTune, Sun Performace Analyzer, DTrace
Benchmarking and profiling are two essential practices for finding bottlenecks
A benchmark measures your system's performance. This can help determine a system's capacity, show you which changes matter and which don't, or show how your application performs with different data
Profiling helps you find where your application spends the most time or consumes the most resources
Benchmarking answers the question "How well does this perform?" and profiling answers the question "Why does it perform the way it does?"
Why Benchmark?
Benchmarks can help you do:
Measure how your application currently performs
Validate your system's scalability
Plan for growth
Test your appplication's ability to tolerate a changing environment
Test different hardware, software, and operating system ocnfigurations
Benchmarking Strategies
There are two primary benchmarking strategies: you can benchmark the appplication as a whole, or isolation MySQL
These two strategies are known as full-stack and single-component benchmarking respectively
What to measure:
Transactions per time unit, Response time or latency, Scalability, Concurrency
Benchmarking Tactics
The first step in planning a benchmark is to identify the problem and the goal
Try to record as much additional information as you can during the benchmarks, such as CPU usage, disk I/O, and network traffic statistics, counters from SHOW GLOBAL STATUS
The best way to get accurate results is to design your benchmark to answer the question you want to answer
It's usually a good idea to automate the benchmark runs
Benchmarking Tools
Full-Stack Tools:
ab
ab is a well-known Apache HTTP server benchmarking tool, it shows how many requests per second your HTTP server is capable of serving
http_load
Similar tool in concept to ab, it's also designed to load a web server, but more flexible
JMeter
A Java application that can load another application and measure its performance, much more complex
Single-Component Tools:
mysqlslap
mysqlslap simulates load on the server and reports timing information
sysbench
sysbench is a multithreaded system benchmarking tool
You can measure the performance of file I/O, the OS scheduler, memory allocation and transfer speed, POSIX threds, and the database server itself
sysben supports scription in the Lua language, which makes it very flexible for testing a variety of scenarios
Database Test Suite
a test kit for running benchmarks similar to some industry-standard benchmarks, such as those published by TPC
MySQL Benchmark Suite
MySQL's own benchmark suite, single-threaded and measure how quickly the server executes queries
Super Smack
a benchmarking, stress-testing, and load-generating tool for MySQL and PostgreSQL
It's a complex, powerful tool that lets you simulate multiple users, load test data into the database, and populate tables with randomly generated data
Benchmarking Examples
$ http_load -parallel 1 -seconds 10 urls.txt $ http_load -paraleel 5 -seconds 10 urls.txt $ http_load -rate 5 -seconds 10 urls.txt $ http_load -rate 20 -seconds 10 urls.txt $ sysbench --test=cpu --cpu-max-prime=20000 run $ sysbench --test=fileio --file-total-size=150G prepare $ sysbench --test=fileio --file-total-size=150G --file-test-mode=rndrw --init-rnd=on --max-time=300 --max-requests=0 run $ sysbench --test=fileio --file-total-size=150G cleanup $ sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=test --mysql-user=root prepare $ sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=test --mysql-user=root --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run sql-bench$ ./run-all-tests --server=mysql --user=root --log --fast sql-bench$ ./test-insert
Profiling
Database access if often the bottleneck in application
Bottlenecks can also be caused by any of the following:
External resources, such as calls to web services or search engines
Operations that require processing large amounts of data in the application, such as parsing big XML files
Expensive operations in tight loops, such as abusing regular expressiosn
Badly optimized algorithms, such as naive search algorithms to find items in lists
Application profiling can help you find the bottlenecks, and it's an important step in monitoring and improving overall performance
We recommend that you include profiling code in every new project you start
Your profiling code should gather and log at least the following:
Total execution time, or "wall-clock time"(in web applications, this is the total page render time)
Each query executed, and its execution time
Each connection opened to the MySQL server
Every call to an external resource, such as web services, memcached, and externally invoked scripts
Potentially expensive function calls, such as XML parsing
User and system CPU time
MySQL has two kinds of query logs: the general log and the slow log
The general log writes out every query as the server receives it, even some events such as connecting and disconnecting and queries may not even be executed due to errors
log = <file_name>
The slow log contains only queries that have executed
In particlular, slow log logs queries that take more than a specified amount of time to execute
log-slow-queries = <file_name> long_query_time = 2 log-queries-not-using-indexex log-slow-admin-statements
Log analysis tools:
mysqldumpslow
mysql_slow_log_filter
mysql_slow_log_parser
mysqlsla
SHOW STATUS/SHOW PROFILE
Operating System Profiling
netstat
vmstat
iostat
mpstat
strace
OProfile
pgrof
Intel VTune, Sun Performace Analyzer, DTrace
评论
1 楼
hideto
2009-07-02
FW也是在Rails程序里有Profiling相关的代码的,每天发一个TOP 20 slow urls和TOP 20 slow queries的report出来。其实这个可以做成一个通用的插件的,不知道有没有现成的
发表评论
-
HPM Note5, Query Performance Optimization
2009-07-21 18:05 1440Slow Query Basics: Optimize Dat ... -
HPM Note4, Schema Optimization and Indexing
2009-07-16 18:04 1437Choosing Optimal Data Types Sma ... -
HPM Note2, MySQL Architecture
2009-06-30 17:13 1726MySQL's Logical Architecture Th ... -
HPM Note1,Book Organization
2009-06-23 09:49 1591How This Book Is Organization ... -
MySQL Architecture
2009-01-18 00:12 3060MySQL Core Modules: Server In ... -
MySQL优化笔记
2008-10-28 17:59 3417MySQL 5.1参考手册 :: 7. 优化 一、查询优化 ... -
MySQL里获取当前week、month、quarter的start_date/end_date
2008-10-21 14:14 7411当前week的第一天: select date_sub(cur ... -
mysql里找出一个表的主键被谁作为外键约束
2008-08-13 17:16 2154SELECT ke.referenced_table_n ... -
SQL性能调优:2.1 排序的一般性调优
2008-08-05 10:21 3540影响排序速度的原因(从大到小): 1,选择的行数 2,ORDE ... -
TCP-IP详解笔记1.5 RARP:逆地址解析协议
2008-07-25 14:05 2240from http://www.beyondrails.com ... -
SQL性能调优:1.3 其他语法调优
2008-07-25 13:38 1351from http://www.beyondrails.com ... -
SQL性能调优:1.2 特别语法调优
2008-07-24 12:15 2668from http://www.beyondrails.com ... -
SQL性能调优:1.1 一般性语法调优
2008-07-23 14:47 2476from http://www.beyondrails.com ... -
MySQL存储程序开发最佳实践
2008-05-28 13:56 1664MySQL存储程序开发最佳 ... -
MySQL join的文章
2008-05-28 13:00 1540MySQL的联结(Join)语法 -
MySQL索引系列文章
2008-05-28 12:51 1439MySQL索引使用 MySQL索引 MySQL 5.1参考手册 ... -
MySQL存储程序权限控制
2008-05-28 12:29 1368MySQL存储程序权限控制 MySQL5.0引入了一些管理存 ... -
MySQL的Stored Function和Trigger
2008-05-27 18:58 3083MySQL的Stored Function和Trigger ... -
MySQL内建Function
2008-05-22 17:25 6452MySQL内建Function 在MySQL存储程序(存储过 ... -
MySQL存储过程之事务管理
2008-05-21 14:36 44842MySQL存储过程之事务管理 ACID:Atomic、Con ...
相关推荐
HPM1210GC.DLL
上海先楫HPM6750代码工程,支持HPM6750和HPM6450处理器。资源代码可直接编译、运行。
HPM1210.INF
HPM1210SU.ent HPM1210SU.ent HPM1210SU.ent
HPM1005驱动程序 ,是最新的官方驱动程序祝下载的朋友工作愉快
HPM1210SM.exe
HPM1210SU.VER
HPM升级流程协议
hpm1210sd.rar缺的可以用
HPM1210PP.dll
win7 可以使用 hpm10005win7的驱动
HPM1210FPSD.DLL
适用于XP32位操作系统的HPm1005打印机驱动
HPM.1 Firmware Update Open Compute Summit - Compute Summit HPM.1 Firmware Update Engineering Workshop HPM.1 File Format File is not sent as a single image Each action is sent by itself to IPM ...
上海先楫HPM6750官网代码工程,支持HPM6750和HPM6450处理器。资源代码可直接编译、运行。
上海先楫HPM6750代码工程,支持HPM6750和HPM6450处理器。资源代码可直接编译、运行。
上海先楫HPM6750代码工程,支持HPM6750和HPM6450处理器。资源代码可直接编译、运行。
hpm utils for hercws
HPM280_M281_20200612固件硒鼓,降级处理HPM280_M281_20200612固件硒鼓,降级处理HPM280_M281_20200612固件硒鼓,降级处理,HPM280_M281_20200612固件硒鼓,降级处理,不会使用的同学,请下载后自行百度搜索使用方法...
上海先楫HPM6750代码工程,支持HPM6750和HPM6450处理器。资源代码可直接编译、运行。