我们的Rails程序,在使用SpiderDB是,一旦对多个分区节点数据库进行更新,就会发生Mysql::Error: This xid is already exist: COMMIT 错误,在网上查找了很久,中日英三文呀,就没有解决办法。
没有办法,只好试探了。在一个Spider的说明文档中,发现Spider引擎有个参数spider_internal_xa,当设成有效时,可以自动把Transaction转换成xa,这样对于应用程序来说,就不用把原来的Transaction
mysql> BEGIN;
mysql> INSERT, UPDATE, DELETE...
mysql> COMMIT;
改成xa Transaction
mysql> XA START 'xatest';
mysql> INSERT, UPDATE, DELETE...
mysql> XA END 'xatest';
mysql> XA PREPARE 'xatest';
mysql> XA COMMIT 'xatest';
了,是非常方便的。
抱着死马当活马医的心情,设置了spider_internal_xa=1、重新启动SpiderDB后,发现一旦执行INSERT,Spider引擎就会当掉,而且无法重启。错误信息如下:
引用
100703 13:37:15 mysqld_safe Starting mysqld daemon with databases from /root/sandboxes/spider_main/data
100703 13:37:15 [Warning] The syntax '--default-character-set' is deprecated and will be removed in a future release. Please use --character-set-server instead.
100703 13:37:17 InnoDB: Log file ./ib_logfile0 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile0 size to 128 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
100703 13:37:23 InnoDB: Log file ./ib_logfile1 did not exist: new to be created
InnoDB: Setting log file ./ib_logfile1 size to 128 MB
InnoDB: Database physically writes the file full: wait...
InnoDB: Progress in MB: 100
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
100703 13:37:27 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
100703 13:37:28 InnoDB: Started; log sequence number 0 7608332
100703 13:37:28 [Note] Recovering after a crash using tc.log
100703 13:37:28 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.
key_buffer_size=1048576
read_buffer_size=2097152
max_used_connections=0
max_threads=3000
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 3919505 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
thd: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = (nil) thread_stack 0x30000
/home/spider/mysql-5.1.44/5.1.44/bin/mysqld(my_print_stacktrace+0x22) [0x84bc7a2]
/home/spider/mysql-5.1.44/5.1.44/bin/mysqld(handle_segfault+0x37f) [0x81fbaef]
[0x236420]
/home/spider/mysql-5.1.44/5.1.44/bin/mysqld(TC_LOG_MMAP::open(char const*)+0x435) [0x82a4c25]
/home/spider/mysql-5.1.44/5.1.44/bin/mysqld [0x81fd9bf]
/home/spider/mysql-5.1.44/5.1.44/bin/mysqld(main+0x4c5) [0x8200655]
/lib/libc.so.6(__libc_start_main+0xdc) [0xc7bdec]
/home/spider/mysql-5.1.44/5.1.44/bin/mysqld [0x8131dd1]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
100703 13:37:28 mysqld_safe mysqld from pid file /root/sandboxes/spider_main/data/mysql_sandbox5144.pid ended
开始的时候只能是安装SpiderDB,以前的数据都没了。
经过多次试探,发现删除掉tc.log、ib_logfile0、ib_logfile1 后,就可以重新启动SpiderDB了。
spider_internal_xa=1不行,还有什么办法呢?在Spider的参数说明中,发现一个spider_support_xa。一旦把这个设置成无效,xa Transaction就不起作用了,不过此时internal_xa是有效的。
OK,测试一下,果然对多个分区节点数据库进行更新的操作不会再让Spider当掉了,而且Mysql::Error: This xid is already exist: COMMIT 错误也没有了。
分享到:
相关推荐
J-Spider:是一个完全可配置和定制的Web Spider引擎.你可以利用它来检查网站的错误(内在的服务器错误等),网站内外部链接检查,分析网站的结构(可创建一个网站地图),下载整个Web站点,你还可以写一个JSpider插件来扩展...
C++ js 互相调用 spider monkey
Spider引擎分布式数据库解决方案.docx
多线程抓取指定的URL站点的网页,并实现结构化保存到本地磁盘。 仅为学习,参考用。
抢先式多线程Spider在智能搜索引擎中的实现
PHPDig 是个开源的小型垂直搜索引擎程序,使用PHP编写,这里的是PHPDig核心的抓取和分析的代码。phpdig_spider.rar
python中的函数在spidermonkey中运行: >>> class Foo: ... def hello(self): ... print "Hello, Javascript world!" >>> cx.bind_class(Foo, bind_constructor=True) >>> cx.eval_script("var f = new ...
web spider 的JAVA搜索引擎
c#编写spider
weibo_spider_爬虫python_关键词爬虫_python_python爬虫_spider_源码.rar
SpiderMonkey-JS引擎,msvc6编译的js引擎!供大家使用!
1.Spider,好用的开源爬虫引擎 2.Eclipse项目
Black_Spider_1.7.2 new tools for upload shell and chek vuln
spider.sav
在Scrapy里面,要抓取网站的链接配置、抓取逻辑、解析逻辑里,都是在Spider里面去完成的,在实现Scrapy爬虫项目里面,最核心的就是Spider类了,它定义了如何爬取某个网站的流程和解析方式
spiderMonkey JS源代码 OpenVXI开发中使用
163spider-master
spider.py, spider.py 关于该爬虫( 爬虫有坑。初学时编写,仅供参考)程序运行示例:Spider.py -u url -d深度日志文件默认当前目录,名字:spider。log##主要参考:http://bbs.chinaunix.net
一个Java的网络爬虫,可用于搜索引擎
spider网络爬虫 c++ 实现 采用广度搜索算法获取url