- 浏览: 574247 次
- 性别:
- 来自: 上海
文章分类
- 全部博客 (174)
- JBPM (3)
- WWF (0)
- JavaScript (11)
- J2EE (40)
- OperationSystem (11)
- 数据库 (12)
- CSS (1)
- Ajax (2)
- J2SE (30)
- Tools (10)
- 服务器中间件 (3)
- 异常 (0)
- Flex (5)
- jQuery (11)
- html (9)
- Ejb (1)
- HTML5 Shiv–让该死的IE系列支持HTML5吧 (1)
- Spring (9)
- Quartz (3)
- log4j (1)
- maven (1)
- cpdetector (1)
- JSON (1)
- log4jdbc (1)
- asm (8)
- FusionCharts (1)
- jqplot (1)
- highcharts (1)
- excanvas (1)
- html5 (1)
- jpcap介绍 (1)
- weblogic (3)
- URLURLClassLoader (0)
- URLClassLoader (1)
- ant (2)
- ivy (2)
- nexus (1)
- IT (0)
- LoadRunner (1)
- SCSS (1)
- ruby (1)
- webstorm (1)
- typescript (1)
- Jboss7 (1)
- wildfly (1)
- oracle (5)
- esb (0)
- dubbo (2)
- zookeeper (3)
- eclipse (1)
- Android (2)
- Studio (1)
- Google (1)
- 微信 (1)
- 企业号 (1)
- Linux (13)
- Oracle12c (1)
- Hadoop (1)
- InletexEMC (1)
- Windows (1)
- Netty (3)
- Marshalling (2)
- Protobuf (1)
- gcc (1)
- Git (1)
- GitLab (1)
- shell (2)
- java (3)
- Spring4 (1)
- hibernate4 (1)
- postgresql (1)
- ApacheServer (2)
- Tomcat (2)
- ApacheHttpServer (2)
- realvnc (1)
- redhat (7)
- vncviewer (1)
- LVS (4)
- LVS-DR (1)
- RedHat6.5 (5)
- LVS-NAT (1)
- LVS-IPTUNNEL (2)
- LVS-TUN (1)
- keepalived (2)
- yum (1)
- iso (1)
- VMware (1)
- redhat5 (1)
- ha (1)
- nginx (2)
- proguard (1)
- Mat (1)
- DTFJ (1)
- axis2 (1)
- web service (1)
- centos (1)
- random (1)
- urandom (1)
- apache (1)
- IBM (1)
- cve (1)
- 漏洞 (1)
- JDBC (1)
- DataSource (1)
- jdk (1)
- tuxedo (2)
- wtc (1)
最新评论
-
skying007:
好资料,谢谢分享给啊
FusionCharts在服务器端导出图片(J2EE版) -
cgnnzg:
大神好 可以发一份源码给我学习么 多谢了 978241085 ...
springmvc+dubbo+zookeeper -
jifengjianhao:
求源码:854606899@qq.com
springmvc+dubbo+zookeeper -
wdloyeu:
shihuan8@163.com邮箱网盘在哪,没找到。能给份源 ...
Java Socket长连接示例代码 -
huangshangyuanji:
求代码:45613032@qq.com
springmvc+dubbo+zookeeper
IndexerMmseg4j.java代码如下:
SearcherMmseg4j.java代码如下:
mytestfile.txt内容如下:
import java.io.ByteArrayOutputStream; import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.IOException; import java.io.InputStream; import org.apache.commons.lang.SystemUtils; import org.apache.lucene.analysis.Analyzer; import org.apache.lucene.analysis.cjk.CJKAnalyzer; import org.apache.lucene.analysis.standard.StandardAnalyzer; import org.apache.lucene.document.Document; import org.apache.lucene.document.Field; import org.apache.lucene.document.Field.Index; import org.apache.lucene.document.Field.Store; import org.apache.lucene.index.IndexReader; import org.apache.lucene.index.IndexWriter; import org.apache.lucene.index.IndexWriterConfig; import org.apache.lucene.index.Term; import org.apache.lucene.store.Directory; import org.apache.lucene.store.FSDirectory; import org.apache.lucene.util.Version; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.chenlb.mmseg4j.analysis.SimpleAnalyzer; public class IndexerMmseg4j { private final static Logger logger = LoggerFactory.getLogger(IndexerMmseg4j.class); static Version matchVersion = Version.LUCENE_36; static String indexPath = "D:" + SystemUtils.FILE_SEPARATOR + "contentWindow" + SystemUtils.FILE_SEPARATOR + "index"; static String filePath = "D:" + SystemUtils.FILE_SEPARATOR + "contentWindow" + SystemUtils.FILE_SEPARATOR + "files" + SystemUtils.FILE_SEPARATOR + "mytestfile.txt"; static Analyzer analyzer = new StandardAnalyzer(matchVersion); static Analyzer a3 = new CJKAnalyzer(matchVersion); //二分法分词 static Analyzer a4 = new SimpleAnalyzer(); //中文分词器mmseg4j中提供的一种分词器 public static byte[] getBytesFromFile(File file) { if(file != null){ FileInputStream fis; try { fis = new FileInputStream(file); if(fis != null){ int len = fis.available(); byte[] bytes = new byte[len]; fis.read(bytes); //现在file中的内容全读到了byte[]数组中 return bytes; } } catch (FileNotFoundException e) { logger.error(e.getMessage()); e.printStackTrace(); } catch (IOException e) { logger.error(e.getMessage()); e.printStackTrace(); } } /* if (file == null){ return null; } try { FileInputStream stream = new FileInputStream(file); ByteArrayOutputStream out = new ByteArrayOutputStream(1000); byte[] b = new byte[1000]; int n; while ((n = stream.read(b)) != -1) out.write(b, 0, n); stream.close(); out.close(); return out.toByteArray(); } catch (IOException e){ } */ /* InputStream is; try { is = new FileInputStream(file); long length = file.length(); if (length > Integer.MAX_VALUE) { // File is too large } byte[] bytes = new byte[(int)length]; int offset = 0; int numRead = 0; while (offset < bytes.length && (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) { offset += numRead; } if (offset < bytes.length) { throw new IOException("Could not completely read file " + file.getName()); } is.close(); return bytes; } catch (FileNotFoundException e) { logger.error(e.getMessage()); e.printStackTrace(); } catch (IOException e) { logger.error(e.getMessage()); e.printStackTrace(); } */ return null; } public static byte[] addByte(byte[] array1, byte[] array2) { if(array1.length==0 && array2.length==0){ return null; } byte[] message = new byte[array1.length+array2.length]; for(int i=0; i<message.length; i++){ if(i < array1.length){ message[i] = array1[i]; }else{ message[i] = array2[i-array1.length]; } } return message; } @org.junit.Test public static void testCreateIndex(){ System.out.println(indexPath); System.out.println(filePath); Directory dir; try { dir = FSDirectory.open(new File(indexPath)); Document doc = new Document(); Field titleField = new Field("title", new File(filePath).getName(), Store.YES, Index.ANALYZED); String content = new String(IndexerMmseg4j.getBytesFromFile(new File(filePath)), "GB18030"); /* * Store.YES : 用于存储字段值。适用于显示搜索结果的字段。例如,文件路径和URL * Store.NO : 没有存储字段值。例如,电子邮件消息正文 * Index.ANALYZED : 用于字段索引和分析。例如,电子邮件消息正文和标题 * Index.NO : 适用于未搜索的字段。例如,仅用于存储字段 * Index.NOT_ANALYZED : 用于编制索引但不分析的字段。它在整体中保留字段的原值。例如,日期和个人名称 * */ Field contentField = new Field("content", content, Store.YES, Index.ANALYZED); doc.add(titleField); doc.add(contentField); IndexWriterConfig iwc = new IndexWriterConfig(matchVersion, a4); //默认create_or_append iwc.setOpenMode(IndexWriterConfig.OpenMode.CREATE); //总是重新创建 IndexWriter iw = new IndexWriter(dir, iwc); iw.addDocument(doc); iw.close(); } catch (IOException e) { logger.error(e.getMessage()); e.printStackTrace(); } } @org.junit.Test public static void testDeleteIndex(){ Directory dir; try { dir = FSDirectory.open(new File(indexPath)); IndexReader indexReader = IndexReader.open(dir); indexReader.deleteDocuments(new Term("content", "上海")); indexReader.close(); } catch (IOException e) { logger.error(e.getMessage()); e.printStackTrace(); } } }
SearcherMmseg4j.java代码如下:
import java.io.File; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.IOException; import java.io.StringReader; import java.io.UnsupportedEncodingException; import java.net.URLDecoder; import java.nio.charset.Charset; import org.apache.commons.lang.SystemUtils; import org.apache.lucene.analysis.Analyzer; import org.apache.lucene.analysis.TokenStream; import org.apache.lucene.analysis.cjk.CJKAnalyzer; import org.apache.lucene.analysis.standard.StandardAnalyzer; import org.apache.lucene.analysis.tokenattributes.OffsetAttribute; import org.apache.lucene.analysis.tokenattributes.TermAttribute; import org.apache.lucene.document.Document; import org.apache.lucene.document.Field; import org.apache.lucene.document.Field.Index; import org.apache.lucene.document.Field.Store; import org.apache.lucene.index.IndexReader; import org.apache.lucene.index.IndexWriter; import org.apache.lucene.index.IndexWriterConfig; import org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy; import org.apache.lucene.index.Term; import org.apache.lucene.queryParser.MultiFieldQueryParser; import org.apache.lucene.queryParser.ParseException; import org.apache.lucene.queryParser.QueryParser; import org.apache.lucene.search.FuzzyQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.PrefixQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.ScoreDoc; import org.apache.lucene.search.TermRangeQuery; import org.apache.lucene.search.TopDocs; import org.apache.lucene.search.WildcardQuery; import org.apache.lucene.search.highlight.Formatter; import org.apache.lucene.search.highlight.Fragmenter; import org.apache.lucene.search.highlight.Highlighter; import org.apache.lucene.search.highlight.InvalidTokenOffsetsException; import org.apache.lucene.search.highlight.QueryScorer; import org.apache.lucene.search.highlight.Scorer; import org.apache.lucene.search.highlight.SimpleFragmenter; import org.apache.lucene.search.highlight.SimpleHTMLFormatter; import org.apache.lucene.search.highlight.TokenSources; import org.apache.lucene.store.Directory; import org.apache.lucene.store.FSDirectory; import org.apache.lucene.store.RAMDirectory; import org.apache.lucene.util.AttributeSource; import org.apache.lucene.util.Version; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.chenlb.mmseg4j.analysis.SimpleAnalyzer; public class SearcherMmseg4j { private final static Logger logger = LoggerFactory.getLogger(SearcherMmseg4j.class); static Version matchVersion = Version.LUCENE_36; static String indexPath = "D:" + SystemUtils.FILE_SEPARATOR + "contentWindow" + SystemUtils.FILE_SEPARATOR + "index"; static String filePath = "D:" + SystemUtils.FILE_SEPARATOR + "contentWindow" + SystemUtils.FILE_SEPARATOR + "files" + SystemUtils.FILE_SEPARATOR + "mytestfile.txt"; static Analyzer analyzer = new StandardAnalyzer(matchVersion); static Analyzer a3 = new CJKAnalyzer(matchVersion); //二分法分词 static Analyzer a4 = new SimpleAnalyzer(); //中文分词器mmseg4j中提供的一种分词器 public static byte[] getBytesFromFile(File file) { if(file != null){ FileInputStream fis; try { fis = new FileInputStream(file); if(fis != null){ int len = fis.available(); byte[] bytes = new byte[len]; fis.read(bytes); //现在file中的内容全读到了byte[]数组中 return bytes; } } catch (FileNotFoundException e) { logger.error(e.getMessage()); e.printStackTrace(); } catch (IOException e) { logger.error(e.getMessage()); e.printStackTrace(); } } return null; } @org.junit.Test public static void testSearch(){ Directory dir; try { dir = FSDirectory.open(new File(indexPath), null); IndexSearcher is = new IndexSearcher(dir); System.out.println(is.maxDoc()); String[] fields = {"title", "content"}; QueryParser qp = new MultiFieldQueryParser(matchVersion, fields, a4); // QueryParser qp = new QueryParser(matchVersion, "content", analyzer); // Query query = qp.parse("汉字"); Query query = qp.parse("上海"); // System.out.println(query.toString("content")); /* Query wildcardQuery = new WildcardQuery(new Term("content", "How*")); //实现通配符查询(*表示零个以上,?表示1个以上) Query fuzzyQuery = new FuzzyQuery(new Term("content", "administritor")); //查找与administritor最相近的单词(主要检索拼写错误) Term beginDate = new Term("date", "20120601"); Term endDate = new Term("date", "20120630"); Query rangeQuery = new TermRangeQuery("date", "20120601", "20120630", true, true); //在某个范围内搜索 Query prefixQuery = new PrefixQuery(new Term("content", "Why")); //构造前缀搜索引擎 */ //TopDocs用来封装搜索结果以及ScoreDoc的总数(ScoreDoc: 搜索结果中指向文档的简单指针) TopDocs tDocs = is.search(query, 10000); //一次查询多少个结果 ScoreDoc[] scoreDoc = tDocs.scoreDocs; //获取搜索结果中指向文档的简单指针 //准备高亮器 Formatter formatter = new SimpleHTMLFormatter("<span class=\"highlighter\">", "</span>"); Scorer fragmentScorer = new QueryScorer(query); Highlighter highlighter = new Highlighter(formatter, fragmentScorer); Fragmenter fragmenter = new SimpleFragmenter(100); //高亮范围 highlighter.setTextFragmenter(fragmenter); int numTotalHits = tDocs.totalHits; System.out.println("总共有【" + numTotalHits + "】条结果"); System.out.println(tDocs.scoreDocs.length); //打印ScoreDoc的总数 // int k = tDocs.scoreDocs[0].doc ; //文档内部编号 // Document doc = is.doc(k) ; //更具文档编号取出对应文档 Document doc = is.doc(0); // doc.getField("content"); //获取属性值,与下相同 String content = doc.get("content"); //获取属性值 //如果当前属性值中没有出现关键字, 则返回null String hc = highlighter.getBestFragment(a4, "content", content); System.out.println("hc:" + hc); if(hc == null){ //如果无结果那么返回原文的前50个字符 hc = content.substring(0, Math.min(50,content.length())); // Field contentField=doc.getFieldable("content"); } Field contentField = (Field) doc.getFieldable("content"); contentField.setValue(hc); // doc.getField("content").setValue(hc); TokenStream ts = a4.tokenStream("content", new StringReader(content)); // System.out.println("token: " + ts.getAttribute(String.class).toString()); OffsetAttribute offsetAttribute = ts.getAttribute(OffsetAttribute.class); TermAttribute termAttribute = ts.getAttribute(TermAttribute.class); while (ts.incrementToken()) { int startOffset = offsetAttribute.startOffset(); int endOffset = offsetAttribute.endOffset(); String term = termAttribute.term(); // System.out.println(term); } /* KeepOnlyLastCommitDeletionPolicy kolcdp = new KeepOnlyLastCommitDeletionPolicy(); //删除过时的索引,只保留最后一次提交的索引策略 //下面的句子删除所有的索引 Directory deleteDir = FSDirectory.open(new File(indexPath)); IndexReader indexReader = IndexReader.open(deleteDir); for(int i=0; i<scoreDoc.length; i++){ indexReader.deleteDocument(i); } indexReader.close(); */ } catch (IOException e) { logger.error(e.getMessage()); e.printStackTrace(); } catch (ParseException e) { logger.error(e.getMessage()); e.printStackTrace(); } catch (InvalidTokenOffsetsException e) { logger.error(e.getMessage()); e.printStackTrace(); } } @org.junit.Test public static void testCreateRAMandFS(){ Directory fsDir; try { fsDir = FSDirectory.open(new File(indexPath)); //1.将索引读取到内存中 Directory ramDir = new RAMDirectory(fsDir); //2.填入文档 Document doc = new Document(); Field titleField = new Field("title", new File(filePath).getName(), Store.YES, Index.ANALYZED); String content = new String(SearcherMmseg4j.getBytesFromFile(new File(filePath))); Field contentField = new Field("content", content, Store.YES, Index.ANALYZED); doc.add(titleField); doc.add(contentField); IndexWriterConfig ramiwc = new IndexWriterConfig(matchVersion, analyzer); //默认create_or_append // ramiwc.setOpenMode(IndexWriterConfig.OpenMode.CREATE); //总是重新创建 IndexWriter ramiw = new IndexWriter(ramDir, ramiwc); ramiw.addDocument(doc); ramiw.close(); //3.关闭时,写入到文件 IndexWriterConfig fsiwc = new IndexWriterConfig(matchVersion, analyzer); //默认create_or_append fsiwc.setOpenMode(IndexWriterConfig.OpenMode.CREATE); //总是重新创建 IndexWriter fsiw = new IndexWriter(fsDir, fsiwc); //将内存的索引文件加入到fsiw中 fsiw.addIndexes(ramDir); fsiw.commit(); //优化索引文件(合并索引文件) fsiw.optimize(); fsiw.close(); System.out.println("===执行完毕"); } catch (IOException e) { logger.error(e.getMessage()); e.printStackTrace(); } } /** * http://www.cnblogs.com/java_cSharp/archive/2011/07/17/lucene.html */ public static void main(String[] args) { IndexerMmseg4j.testCreateIndex(); // SearcherMmseg4j.testSearch(); SearcherMmseg4j.testCreateRAMandFS(); } }
mytestfile.txt内容如下:
Why is luanma? 于士博 于时欢 上海 上海滩 北京 北京是首都 北京房价贵 How are you? yes, I am! 1+1=2 2*3=6 @#$%^&
- junit-4.4.jar (159 KB)
- 下载次数: 22
- lucene-analyzers-3.6.0.jar (1.1 MB)
- 下载次数: 46
- lucene-core-3.6.0.jar (1.5 MB)
- 下载次数: 39
- lucene-highlighter-3.6.0.jar (87.1 KB)
- 下载次数: 33
- lucene-memory-3.6.0.jar (29.1 KB)
- 下载次数: 32
- mmseg4j-all-1.8.5.jar (94.7 KB)
- 下载次数: 30
发表评论
-
Spring4Mvc整合Hibernate4框架示例
2016-02-05 11:52 1361今天用NetBeans8.1搭建了一套Spring4MVC+H ... -
Spring4配置properties文件的一点儿细节
2016-02-04 15:40 2759http://repo.spring.io/release/o ... -
JavaWeb发送信息到微信公众平台的企业号
2015-11-01 22:42 3761首先到微信公众平台申请微信企业号: https://qy.we ... -
springmvc+dubbo+zookeeper
2015-09-22 11:34 11922笔者这里使用的是apache- ... -
OracleAS10g安装配置与部署
2015-07-01 20:59 663先到Oracle官方下载OracleAS10的安装包,截图如下 ... -
java和spring使用weblogic数据源配置
2014-10-13 11:29 2948需要weblogic.jar和jrmpclient.jar ... -
一套基本的SpringMvc工程所需要的最基本jar包(pom.xml)
2014-08-19 16:38 1879附件里是pom.xml文件。 -
html的form表单提交得不到参数问题
2014-07-25 16:13 1803今天同事问我他的form表单按submit方式提交,在Java ... -
Spring3Mvc导出PDF的3种形式+导出excel的两种形式(exportAnimal.do)
2014-07-19 22:55 1086附件里SpringMvc导出PDF和Excel的示例代码。 ... -
Java利用中国网建SMS短信通平台发送手机短信
2014-04-26 21:55 2198JAVA发送手机短信,流传有几种方法: (1)使用webser ... -
在Java中发送电子邮件
2014-04-26 18:42 9564要先申请一个网易的126邮箱和一个新浪的com邮箱。 以下程 ... -
cpdetector_1.0.10_binary下载
2014-03-26 20:53 991附件里是cpdetector_1.0.10_binary.ra ... -
在Java代码中用Axis2调用DotNet语言的WebService程序
2014-02-19 15:19 2526package com.shihuan.ymail.com ... -
SpringMvc多语言配置
2014-01-14 16:01 14378applicationContext.xml文件中加入如下代码 ... -
log4j多级别多输出文件的配置
2013-07-22 21:28 1532以下代码需要log4j-1.2.17.jar的支持。 B ... -
mysql和Oracle在对clob和blob字段的处理
2012-08-25 18:50 30847一、MySQL与Oracle数据库如何处理Clob,Bl ... -
Spring3Mvc+Hibernate整合Axis2发布WebService并配置Quartz调度
2012-08-05 12:54 2182附件里是笔者的源代码,相关的jar包在笔者的abcdef的16 ... -
Mina入门程序
2012-06-02 15:05 1118附件里是Mina的入门程序。 http://mina.apa ... -
让Hibernate的执行语句看见完整参数
2012-04-29 10:09 1016http://www.ibm.com/developerwor ... -
ant打包Web工程
2012-03-18 13:13 3453build.properties文件内容如下: project ...
相关推荐
主要介绍了详解SpringBoot+Lucene案例介绍,文中通过示例代码介绍的非常详细,对大家的学习或者工作具有一定的参考学习价值,需要的朋友们下面随着小编来一起学习学习吧
lucene部分案例的源代码
lucene全文检索案例源码 lucene全文检索案例源码
Lucene全文检索案例Lucene全文检索案例Lucene全文检索案例Lucene全文检索案例
使用lucene技术,实现android的本地文档资源的全文检索功能,并实现关键字高亮
关于搜索的一个小案例,使用的是lucene,框架为ibatis,spring,stuts2
lucene的开发案例,是一个比较完整的项目,只需要稍微的修改,就可以在本地建立索引
Lucene的的学习资料及案例,包括一个lucene的学习资料总结。供大家学习使用,也有本人写的一个小案例。
说明: 例子是根据lucene3.6写的,也可以说是直接copy别人的。 包括参考文章的代码,以及修改部分之后的代码
包含最新lucene5.2.1的入门经典案例,以及程序中所需的lucene-core-5.2.1.jar、lucene-analyzers-common-5.2.1.jar等几个jar包。代码可运行。
Lucene搜索案例.rarLucene搜索案例.rarLucene搜索案例.rarLucene搜索案例.rar
lucene案例代码https://blog.csdn.net/u013938578/category_12317662.html
lucene3.5全文检索案例lucene+demo
《Lucene实战 第2版 》基于Apache的Lucene 3 0 从Lucene核心 Lucene应用 案例分析3个方面详细系统地介绍了Lucene 包括认识Lucene 建立索引 为应用程序添加搜索功能 高级搜索技术 扩展搜索 使用Tika提取文本 Lucene...
Lucene 4.7 测试案例,最新版本测试,使用详解
最新的lucene jar及demo和demo详细制作方案。
《Lucene实战(第2版)》基于Apache的Lucene 3.0,从Lucene核心、Lucene应用、案例分析3个方面详细系统地介绍了Lucene,包括认识Lucene、建立索引、为应用程序添加搜索功能、高级搜索技术、扩展搜索、使用Tika提取文本...
完成当前lucene案例必须的jar包,包括commons-io-2.6.jar...
lucene4.8学习资料和案例,自己学习整理。整理了各种查询,包括分页、排序等!对100W,500W,1000W三个级别的数据量分别进行了测试!
lucene 拼音检索 拼音 分词器等实例应用