- 浏览: 1046222 次
- 性别:
- 来自: 杭州
文章分类
最新评论
-
nieanan:
感谢,很有帮助。
eclipse 改变JAVA_HOME路径 -
Orange_killer:
写的什么东西,文不对题
Hibernate Search大数据量手动建立索引 -
xiaoasha:
org.eclipse.equinox.servlet.api ...
《OSGI实战》遇到的问题 -
powertech:
写的挺细,有用!
SyntaxError: Non-ASCII character Python、Unicode和中文 -
huang_yong:
public class XMLUtil {
pri ...
XStream 去除生成的XML节点的class="list"
刚刚看了Hibernate Search,想要写个程序试着使用JPA建立一个索引,由于数据量比较大,所以是用分页来取得数据,但是老是报outOfMemory错误,最后解决了,所以把关键的代码贴出来和大家分享:
public void index(){ EntityManager em = EntityManagerHolder.getEntityManager(); //wrap a EntityManager object Query query = em.createQuery(" from A").setMaxResults(1000); for(int i=0, j=0; i < 1000; i++, j+=1000){ FullTextEntityManager ftem = org.hibernate.search.jpa.Search.getFullTextEntityManager(em); ftem.getTransaction().begin(); @SuppressWarnings("unchecked") List<Item> items = query.setFirstResult(j).getResultList(); for (Item item : items) { ftem.index(item); //manually index an item instance } ftem.flush(); //ftem.clear(); /*必须每次都commit一次,否则会内存溢出*/ ftem.getTransaction().commit(); //index are written at commit time /* 1. EntityManager.clear()必须在ftem.index之后调用, * 否则ftem无法再context中找不到相应的对象。 * * 2. 因为EntityManager中有个Context用于缓存事务中query到 * 以及update,insert到数据库的对象,这样可以减少数据库的读写 * 次数,所以web应用的时候每个请求的线程一个EntityManager * 对象。如果要用同一个EntityManager分页获得大数据量的数据, * 就必须定期clear 一次。否则缓存写满了就会报outOfMemery错误。 * */ em.clear(); } }
EntityManagerHolder:
public class EntityManagerHolder { private static ThreadLocal<EntityManager> entityManagerHolder = new ThreadLocal<EntityManager>(); public static EntityManager getEntityManager() { return entityManagerHolder.get(); } public static void setEntityManager(EntityManager session) { entityManagerHolder.set(session); } }
初始化EntityManagerHolder:
public class TestCase { protected EntityManagerFactory factory; @Before public void setUp() throws Exception { factory = Persistence.createEntityManagerFactory("test"); EntityManagerHolder.setEntityManager(factory.createEntityManager()); } @After public void tearDown() throws Exception { EntityManagerHolder.getEntityManager().close(); EntityManagerHolder.setEntityManager(null); factory.close(); } }
下面是我之前碰到的outOfMemory错误的详细信息:
java.lang.OutOfMemoryError: Java heap space at org.hibernate.util.IdentityMap.entryArray(IdentityMap.java:224) at org.hibernate.util.IdentityMap.concurrentEntries(IdentityMap.java:89) at org.hibernate.event.def.AbstractFlushingEventListener.prepareEntityFlushes(AbstractFlushingEventListener.java:136) at org.hibernate.event.def.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:88) at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:49) at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1027) at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:365) at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:137) at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:54) at com.chaizj.search.proptotype.ManuallyIndex.index(ManuallyIndex.java:26) at com.chaizj.search.proptotype.test.MannuallyIndexTest.postSetUp(MannuallyIndexTest.java:12) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41) at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31) at org.junit.runners.ParentRunner.run(ParentRunner.java:220) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:46) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java.lang.OutOfMemoryError: Java heap space at java.util.HashMap.addEntry(HashMap.java:797) at java.util.HashMap.put(HashMap.java:431) at org.hibernate.engine.StatefulPersistenceContext.addEntity(StatefulPersistenceContext.java:336) at org.hibernate.engine.StatefulPersistenceContext.addEntity(StatefulPersistenceContext.java:431) at org.hibernate.engine.TwoPhaseLoad.addUninitializedEntity(TwoPhaseLoad.java:260) at org.hibernate.loader.Loader.loadFromResultSet(Loader.java:1390) at org.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:1332) at org.hibernate.loader.Loader.getRow(Loader.java:1230) at org.hibernate.loader.Loader.getRowFromResultSet(Loader.java:603) at org.hibernate.loader.Loader.doQuery(Loader.java:724) at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:259) at org.hibernate.loader.Loader.doList(Loader.java:2228) at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2125) at org.hibernate.loader.Loader.list(Loader.java:2120) at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:401) at org.hibernate.hql.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:361) at org.hibernate.engine.query.HQLQueryPlan.performList(HQLQueryPlan.java:196) at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1148) at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102) at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:67) at com.chaizj.search.proptotype.ManuallyIndex.index(ManuallyIndex.java:21) at com.chaizj.search.proptotype.test.MannuallyIndexTest.postSetUp(MannuallyIndexTest.java:12) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
评论
3 楼
Orange_killer
2014-09-17
写的什么东西,文不对题
2 楼
jespring
2010-06-21
你这个标题不对,让人误解以为是hibernate search呢?
1 楼
jespring
2010-06-21
为什么我报
javax.persistence.PersistenceException: No Persistence provider for EntityManager named test
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:89)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:60)
javax.persistence.PersistenceException: No Persistence provider for EntityManager named test
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:89)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:60)
发表评论
-
SelectionKey.attachment() returns null
2011-09-17 15:49 5703写NIO程序的时候遇到了SelectionKey.attach ... -
Eclipse远程调试Weblogic运行的源代码
2010-11-25 20:53 1678看过许多远程调试的 ... -
jsp:include 找不到*.do的解决方法
2010-11-15 16:18 3969由于要兼容以前的一个系统,不得已自己实现了一个filter,通 ... -
ArrayList from HashMap values
2010-10-09 14:47 1536Is there a way to cast HashMap ... -
hibernate mapping boolean field
2010-09-15 16:34 1315有一个属性: private boolean isPe ... -
LinkedHashMap测试
2010-09-13 10:26 1415Map 接口的哈希表和链接列表实现,具有可预知的迭代顺序 ... -
Myeclipse 8.5 配置SVN
2010-09-13 10:25 4319很早以前用过VSS的人都知道,相比之下SVN的确是个好东东。他 ... -
Myeclipse配置插件
2010-09-13 10:24 1367我们在用Myeclipse进行开 ... -
Java 调用 Eclipse的编译器 JDT
2010-08-31 11:48 3085Java 调用 Eclipse的编译器 JDT 想给项 ... -
java.lang.OutOfMemoryError: PermGen space及其解决方法(转)
2010-08-05 14:45 1366java.lang.OutOfMemoryError: P ... -
如何得到JDBC Insert 语句执行后插入Oracle 数据库记录的主键
2010-07-15 17:18 2031在应用中,很多时候会对表的主键用一个自动增涨的数来付值,如Or ... -
让iBatis中的insert返回主键 [ibatis insert 返回 自增主键]
2010-07-13 23:05 3803让iBatis中的insert返回主键 [ibatis i ... -
eclipse 改变JAVA_HOME路径
2010-07-12 12:54 15370让eclipse.ini中-vm选项指向一个已安装的jdk并且 ... -
Tomcat6 class load order
2010-05-20 15:47 1173Therefore, from the perspective ... -
修改Java系统默认编码
2010-05-10 14:56 9082通过以下代码可以修改动态当前正在运行的java系统默认编码格式 ... -
java编码格式总结
2010-05-07 13:46 4136乱码一直是一个很让人头疼的问题,这次往oracle的Blob里 ... -
SWT Table单选按钮实现
2010-05-03 01:01 3413table_short = new Table (compos ... -
eclipse 下tomcat出现did not find a matching property
2010-04-15 15:15 1915Software : apache-tomcat-6 ... -
WTP配置JNDI
2010-03-17 10:54 1289想要在WTP环境下面配置JNDI,有两种方式,一种是在Ser ... -
Stateless & Stateful Knowledge Session
2010-03-08 14:06 1413详细请看: http://hudson.jboss.org/ ...
相关推荐
使用Hibernate将大量记录插入到数据库
Hibernate Search is a library providing full-text search capabilities to Hibernate. It opens doors to more human friendly and efficient search engines while still following the Hibernate and Java ...
hibernate search全文索引的创建与查询实例 hibernate-search-analyzers-4.1.1.Final.jar
它是hibernate对著名的全文检索系统Lucene的一个集成方案,作用在于对数据表中某些内容庞大的字段(如声明为text的字段)建立全文索引,这样通过hibernate search就可以对这些字段进行全文检索后获得相应的POJO,...
2 手动生成索引 3 文档上传(自动建立索引) 4 文档更新(自动建立索引) 5 文档删除 使用说明: 1 需要先恢复数据库 searchDB_2008sqlserver.bak 2 建立upload目录 本例子在 E:\Tomcat50\upload 3 注意参数正确配置...
hibernate-search, Hibernate Search Hibernate 搜索版本:5.8.0. Final - 13-09-2017描述针对Java对象的全文搜索这个项目提供 Hibernate ORM和全文索引服务( 如 Apache Lucene和 Elasticsearch
hibernate Search in action
使用 hibernate-search 实现全文检索及文档管理的例子 1 全文检索 2 手动重新建立索引 3 文档上传(自动建立索引) 4 文档更新(自动建立索引) 5 文档删除
hibernate search 开发环境的搭建。
项目中使用Hibernate进行大数据量的性能测试,有一些总结, 1) 在处理大数据量时,会有大量的数据缓冲保存在Session的一级缓存中 2) 对大数据量查询时,慎用list()或者iterator()返回查询结果, 3) 对于关联操作...
hibernate search 里面有关于代码的详细说明 请看readme txt 有相关jar直接运行
Hibernate Search 4.4.0.Final API 帮助文档
HibernateSearch的使用方法
hibernateSearch 列子源代码
hibernate_search.pdf
基于Spring的Hibernate Search全文检索功能示例 实例下载
压缩包包括项目完整代码,详细说明和项目结构图,效果图 项目实现了分页和高亮显示 MAVEN项目:HibernateSearchDemo IDE :MyEclipse jdk :1.6 数据库 :MySql ...hibernate search 版本 4.4.1.Final
hibernate-search-3.4.1 api chm
Hibernate Search in Action introduces both the principles of enterprise search and the implementation details a Java developer will need to use Hibernate Search effectively. This book blends the ...