`

javac的词法分析,转载自:百度文库

    博客分类:
  • Java
 
阅读更多
public class Mess {

	/*-------------------ScannerFactory class-----------------------*/
	final Names names;
	final Source source;
	final Keywords keywords;

	/** Create a new scanner factory. */
	protected ScannerFactory(Context context) {
		context.put(scannerFactoryKey, this);
		this.log = Log.instance(context);
		this.names = Names.instance(context);/*---first, add name in the Names to table---*/
		this.source = Source.instance(context);
		this.keywords = Keywords.instance(context);/*---then, add Token constant to table---*/
	}

	/*----------------Scanner class---------------------------------*/
	/** The name table. */
	private final Names names;

	/** The keyword table. */
	private final Keywords keywords;

	/** Common code for constructors. */
	private Scanner(ScannerFactory fac) {
		log = fac.log;
		names = fac.names;
		keywords = fac.keywords;
		source = fac.source;
		allowBinaryLiterals = source.allowBinaryLiterals();
		allowHexFloats = source.allowHexFloats();
		allowUnderscoresInLiterals = source.allowUnderscoresInLiterals();
	}

	/*----------------Names class----------------------------------*/
	public static final Context.Key<Names> namesKey = new Context.Key<Names>();

	public static Names instance(Context context) {
		Names instance = context.get(namesKey);
		if (instance == null) {
			instance = new Names(context);
			context.put(namesKey, instance);
		}
		return instance;
	}

	public final Name slash;
	public final Name hyphen;
	public final Name T;
	public final Name slashequals;
	public final Name deprecated;
	public final Name init;
	public final Name clinit;
	public final Name error;
	public final Name any;
	public final Name empty;
	public final Name one;
	public final Name period;
	public final Name comma;
	public final Name semicolon;
	public final Name asterisk;
	public final Name _this;
	public final Name _super;
	public final Name _default;
	public final Name _class;
	public final Name java_lang;
	public final Name java_lang_Object;
	public final Name java_lang_Class;
	public final Name java_lang_Cloneable;
	public final Name java_io_Serializable;
	public final Name serialVersionUID;
	public final Name java_lang_Enum;
	public final Name java_lang_invoke_MethodHandle;
	public final Name package_info;
	public final Name ConstantValue;
	public final Name LineNumberTable;
	public final Name LocalVariableTable;
	public final Name LocalVariableTypeTable;
	public final Name CharacterRangeTable;
	public final Name StackMap;
	public final Name StackMapTable;
	public final Name SourceID;
	public final Name CompilationID;
	public final Name Code;
	public final Name Exceptions;
	public final Name SourceFile;
	public final Name InnerClasses;
	public final Name Synthetic;
	public final Name Bridge;
	public final Name Deprecated;
	public final Name Enum;
	public final Name _name;
	public final Name Signature;
	public final Name Varargs;
	public final Name Annotation;
	public final Name RuntimeVisibleAnnotations;
	public final Name RuntimeInvisibleAnnotations;
	public final Name RuntimeVisibleTypeAnnotations;
	public final Name RuntimeInvisibleTypeAnnotations;
	public final Name RuntimeVisibleParameterAnnotations;
	public final Name RuntimeInvisibleParameterAnnotations;
	public final Name Value;
	public final Name EnclosingMethod;
	public final Name desiredAssertionStatus;
	public final Name append;
	public final Name family;
	public final Name forName;
	public final Name toString;
	public final Name length;
	public final Name valueOf;
	public final Name value;
	public final Name getMessage;
	public final Name getClass;
	public final Name TYPE;
	public final Name TYPE_USE;
	public final Name TYPE_PARAMETER;
	public final Name FIELD;
	public final Name METHOD;
	public final Name PARAMETER;
	public final Name CONSTRUCTOR;
	public final Name LOCAL_VARIABLE;
	public final Name ANNOTATION_TYPE;
	public final Name PACKAGE;
	public final Name SOURCE;
	public final Name CLASS;
	public final Name RUNTIME;
	public final Name Array;
	public final Name Method;
	public final Name Bound;
	public final Name clone;
	public final Name getComponentType;
	public final Name getClassLoader;
	public final Name initCause;
	public final Name values;
	public final Name iterator;
	public final Name hasNext;
	public final Name next;
	public final Name AnnotationDefault;
	public final Name ordinal;
	public final Name equals;
	public final Name hashCode;
	public final Name compareTo;
	public final Name getDeclaringClass;
	public final Name ex;
	public final Name finalize;
	public final Name java_lang_AutoCloseable;
	public final Name close;
	public final Name addSuppressed;

	public final Name.Table table;

	public Names(Context context) {
		Options options = Options.instance(context);
		table = createTable(options);

		slash = fromString("/");
		hyphen = fromString("-");
		T = fromString("T");
		slashequals = fromString("/=");
		deprecated = fromString("deprecated");

		init = fromString("<init>");
		clinit = fromString("<clinit>");
		error = fromString("<error>");
		any = fromString("<any>");
		empty = fromString("");
		one = fromString("1");
		period = fromString(".");
		comma = fromString(",");
		semicolon = fromString(";");
		asterisk = fromString("*");
		_this = fromString("this");
		_super = fromString("super");
		_default = fromString("default");

		_class = fromString("class");
		java_lang = fromString("java.lang");
		java_lang_Object = fromString("java.lang.Object");
		java_lang_Class = fromString("java.lang.Class");
		java_lang_Cloneable = fromString("java.lang.Cloneable");
		java_io_Serializable = fromString("java.io.Serializable");
		java_lang_Enum = fromString("java.lang.Enum");
		java_lang_invoke_MethodHandle = fromString("java.lang.invoke.MethodHandle");
		package_info = fromString("package-info");
		serialVersionUID = fromString("serialVersionUID");
		ConstantValue = fromString("ConstantValue");
		LineNumberTable = fromString("LineNumberTable");
		LocalVariableTable = fromString("LocalVariableTable");
		LocalVariableTypeTable = fromString("LocalVariableTypeTable");
		CharacterRangeTable = fromString("CharacterRangeTable");
		StackMap = fromString("StackMap");
		StackMapTable = fromString("StackMapTable");
		SourceID = fromString("SourceID");
		CompilationID = fromString("CompilationID");
		Code = fromString("Code");
		Exceptions = fromString("Exceptions");
		SourceFile = fromString("SourceFile");
		InnerClasses = fromString("InnerClasses");
		Synthetic = fromString("Synthetic");
		Bridge = fromString("Bridge");
		Deprecated = fromString("Deprecated");
		Enum = fromString("Enum");
		_name = fromString("name");
		Signature = fromString("Signature");
		Varargs = fromString("Varargs");
		Annotation = fromString("Annotation");
		RuntimeVisibleAnnotations = fromString("RuntimeVisibleAnnotations");
		RuntimeInvisibleAnnotations = fromString("RuntimeInvisibleAnnotations");
		RuntimeVisibleTypeAnnotations = fromString("RuntimeVisibleTypeAnnotations");
		RuntimeInvisibleTypeAnnotations = fromString("RuntimeInvisibleTypeAnnotations");
		RuntimeVisibleParameterAnnotations = fromString("RuntimeVisibleParameterAnnotations");
		RuntimeInvisibleParameterAnnotations = fromString("RuntimeInvisibleParameterAnnotations");
		Value = fromString("Value");
		EnclosingMethod = fromString("EnclosingMethod");

		desiredAssertionStatus = fromString("desiredAssertionStatus");

		append = fromString("append");
		family = fromString("family");
		forName = fromString("forName");
		toString = fromString("toString");
		length = fromString("length");
		valueOf = fromString("valueOf");
		value = fromString("value");
		getMessage = fromString("getMessage");
		getClass = fromString("getClass");

		TYPE = fromString("TYPE");
		TYPE_USE = fromString("TYPE_USE");
		TYPE_PARAMETER = fromString("TYPE_PARAMETER");
		FIELD = fromString("FIELD");
		METHOD = fromString("METHOD");
		PARAMETER = fromString("PARAMETER");
		CONSTRUCTOR = fromString("CONSTRUCTOR");
		LOCAL_VARIABLE = fromString("LOCAL_VARIABLE");
		ANNOTATION_TYPE = fromString("ANNOTATION_TYPE");
		PACKAGE = fromString("PACKAGE");

		SOURCE = fromString("SOURCE");
		CLASS = fromString("CLASS");
		RUNTIME = fromString("RUNTIME");

		Array = fromString("Array");
		Method = fromString("Method");
		Bound = fromString("Bound");
		clone = fromString("clone");
		getComponentType = fromString("getComponentType");
		getClassLoader = fromString("getClassLoader");
		initCause = fromString("initCause");
		values = fromString("values");
		iterator = fromString("iterator");
		hasNext = fromString("hasNext");
		next = fromString("next");
		AnnotationDefault = fromString("AnnotationDefault");
		ordinal = fromString("ordinal");
		equals = fromString("equals");
		hashCode = fromString("hashCode");
		compareTo = fromString("compareTo");
		getDeclaringClass = fromString("getDeclaringClass");
		ex = fromString("ex");
		finalize = fromString("finalize");

		java_lang_AutoCloseable = fromString("java.lang.AutoCloseable");
		close = fromString("close");
		addSuppressed = fromString("addSuppressed");
	}

	/*---------------Keywords class------------------------*/
	public static final Context.Key<Keywords> keywordsKey = new Context.Key<Keywords>();

	public static Keywords instance(Context context) {
		Keywords instance = context.get(keywordsKey);
		if (instance == null)
			instance = new Keywords(context);
		return instance;
	}

	private final Names names;

	protected Keywords(Context context) {
		context.put(keywordsKey, this);
		names = Names.instance(context);

		for (Token t : Token.values()) {
			if (t.name != null)
				enterKeyword(t.name, t);
			else
				tokenName[t.ordinal()] = null;
		}

		key = new Token[maxKey + 1];
		for (int i = 0; i <= maxKey; i++)
			key[i] = IDENTIFIER;
		for (Token t : Token.values()) {
			if (t.name != null)
				key[tokenName[t.ordinal()].getIndex()] = t;
		}
	}

	public Token key(Name name) {
		return (name.getIndex() > maxKey) ? IDENTIFIER : key[name.getIndex()];
	}

	/**
	 * Keyword array. Maps name indices to Token.
	 */
	private final Token[] key;

	/**
	 * The number of the last entered keyword.
	 */
	private int maxKey = 0;

	/**
	 * The names of all tokens.
	 */
	private Name[] tokenName = new Name[Token.values().length];

	private void enterKeyword(String s, Token token) {
		Name n = names.fromString(s);
		tokenName[token.ordinal()] = n;
		if (n.getIndex() > maxKey)
			maxKey = n.getIndex();
	}

	/*-------------------Token enum-----------------------*/
	public enum Token implements Formattable {
		EOF, ERROR, IDENTIFIER, ABSTRACT("abstract"), ASSERT("assert"), 
		BOOLEAN("boolean"), BREAK("break"), BYTE("byte"), CASE("case"), 
		CATCH("catch"), CHAR("char"), CLASS("class"), CONST("const"), 
		CONTINUE("continue"), DEFAULT("default"), DO("do"), 
		DOUBLE("double"), ELSE("else"), ENUM("enum"), EXTENDS("extends"), 
		FINAL("final"), FINALLY("finally"), FLOAT("float"), FOR("for"), 
		GOTO("goto"), IF("if"), IMPLEMENTS("implements"), IMPORT("import"), 
		INSTANCEOF("instanceof"), INT("int"), INTERFACE("interface"), 
		LONG("long"), NATIVE("native"), NEW("new"), PACKAGE("package"), 
		PRIVATE("private"), PROTECTED("protected"), PUBLIC("public"), 
		RETURN("return"), SHORT("short"), STATIC("static"), 
		STRICTFP("strictfp"), SUPER("super"), SWITCH("switch"), 
		SYNCHRONIZED("synchronized"), THIS("this"), THROW("throw"), 
		THROWS("throws"), TRANSIENT("transient"), TRY("try"), 
		VOID("void"), VOLATILE("volatile"), WHILE("while"), INTLITERAL, 
		LONGLITERAL, FLOATLITERAL, DOUBLELITERAL, CHARLITERAL, 
		STRINGLITERAL, TRUE("true"), FALSE("false"), NULL("null"), 
		LPAREN("("), RPAREN(")"), LBRACE("{"), RBRACE("}"), LBRACKET("["), 
		RBRACKET("]"), SEMI(";"), COMMA(","), DOT("."), ELLIPSIS("..."), 
		EQ("="), GT(">"), LT("<"), BANG("!"), TILDE("~"), QUES("?"), 
		COLON(":"), EQEQ("=="), LTEQ("<="), GTEQ(">="), BANGEQ("!="), 
		AMPAMP("&&"), BARBAR("||"), PLUSPLUS("++"), SUBSUB("--"), 
		PLUS("+"), SUB("-"), STAR("*"), SLASH("/"), AMP("&"), BAR("|"), 
		CARET("^"), PERCENT("%"), LTLT("<<"), GTGT(">>"), GTGTGT(">>>"), 
		PLUSEQ("+="), SUBEQ("-="), STAREQ("*="), SLASHEQ("/="), 
		AMPEQ("&="), BAREQ("|="), CARETEQ("^="), PERCENTEQ("%="), 
		LTLTEQ("<<="), GTGTEQ(">>="), GTGTGTEQ(">>>="), MONKEYS_AT("@"), 
		CUSTOM;
	}

	/*----------------SharedNameTable class--------------------*/
	public class SharedNameTable extends Name.Table {

		/**
		 * The hash table for names.
		 */
		private NameImpl[] hashes;

		/**
		 * The number of filled bytes in `names'.
		 */
		private int nc = 0;

		@Override
		public Name fromChars(char[] cs, int start, int len) {
			int nc = this.nc;
			byte[] bytes = this.bytes;
			while (nc + len * 3 >= bytes.length) {
				// System.err.println("doubling name buffer of length " + names.length + " to fit " + len + " chars");//DEBUG
				byte[] newnames = new byte[bytes.length * 2];
				System.arraycopy(bytes, 0, newnames, 0, bytes.length);
				bytes = this.bytes = newnames;
			}
			int nbytes = Convert.chars2utf(cs, start, bytes, nc, len) - nc;
			int h = hashValue(bytes, nc, nbytes) & hashMask;
			NameImpl n = hashes[h];
			while (n != null && (n.getByteLength() != nbytes || !equals(bytes, n.index, bytes, nc, nbytes))) {
				n = n.next;
			}
			if (n == null) {
				n = new NameImpl(this);
				n.index = nc;/*---this is the index of name---*/
				n.length = nbytes;
				n.next = hashes[h];
				hashes[h] = n;/*---save the name in hashes array---*/
				this.nc = nc + nbytes;
				if (nbytes == 0) {
					this.nc++;/*---index increase by 1---*/
				}
			}
			return n;
		}

		static class NameImpl extends Name {
			int index;

			@Override
			public int getIndex() {
				return index;
			}

		}

	}

}




转载地址:http://wenku.baidu.com/view/6f5fa9718e9951e79b892736.html
http://www.docjar.com/html/api/com/sun/tools/javac/parser/ScannerFactory.java.html
http://www.docjar.com/html/api/com/sun/tools/javac/parser/Scanner.java.html
http://www.docjar.com/html/api/com/sun/tools/javac/parser/Keywords.java.html
http://www.docjar.com/html/api/com/sun/tools/javac/util/Names.java.html
http://www.docjar.com/html/api/com/sun/tools/javac/parser/Token.java.html
http://www.docjar.com/html/api/com/sun/tools/javac/util/SharedNameTable.java.html
分享到:
评论

相关推荐

    java写的C-词法分析

    Java编写的C-语言词法分析器是一种工具,它用于解析C-语言源代码,将源代码分解成一系列有意义的符号或标记,这个过程被称为词法分析或扫描。C-语言是一种广泛使用的编程语言,而Java则是一种面向对象的、跨平台的...

    词法分析(java)

    在Java中,词法分析是通过Java编译器(javac)中的词法分析器来实现的。这个阶段的主要任务是识别源代码中的语法规则,并将其转化为便于进一步解析和编译的形式。词法分析器根据预定义的规则(正则表达式或词法规范...

    java解释器初步之词法分析

    在实际编程中,词法分析通常由专门的库或工具完成,如Java的`javac`编译器就内置了词法分析器。但了解如何自己实现一个词法分析器有助于深入理解编译原理和Java语法的细节。 总结来说,这个"java解释器初步之词法...

    使用Java语言编写的词法分析器

    词法分析器,也称为扫描器或标记器,是编译器前端的重要组成部分,它负责将源代码文本转换成一系列的标记(tokens),这些标记代表了程序中的基本元素,如关键字、标识符、运算符、常量等。在Java编程语言中,构建一...

    java语言编写词法、语法、语义分析器

    例如,`javac`编译器中的词法分析器会识别出"int"作为关键字,"myVariable"作为标识符,"5"作为整型常量。 **语法分析器(Parser)** 语法分析器接收词法分析器产生的词法单元流,并根据预定义的语法规则(通常表示...

    编译原理大作业:词法分析器、语法分析器,java实现.zip

    在这个“编译原理大作业:词法分析器、语法分析器,java实现.zip”压缩包中,我们可以看到一个大学生在学习Java编程时对编译原理的应用实践。下面将详细介绍词法分析器和语法分析器的概念以及它们在Java中的实现。 ...

    javac源码和运行说明文件.zip

    - **词法分析**:javac首先将源代码分解成一个个小的标记,这些标记代表了语言中的关键字、标识符、常量等。 - **语法分析**:接着,编译器将标记组合成抽象语法树(AST),验证代码的语法规则。 - **语义分析**...

    javac源码免费下载

    1. **词法分析**:javac首先进行词法分析,将源代码分割成一个个的词法单元(如标识符、关键字、操作符等)。这个过程由`com.sun.source.util.Scanner`类来完成,它通过扫描源文件并生成Token流。 2. **语法分析**...

    javac 源码 javac 源码

    - **词法分析**:javac 首先将源代码分解成一个个称为标记的单元,例如关键字、标识符、常量等。 - **语法分析**:接着,它会根据 Java 语法规则构建抽象语法树(AST),这个树状结构代表了程序的结构。 - **语义...

    javac-ast-experiments:各种AST级`javac`编译器插件的示例和实验

    javac-ast-experiments 本项目用于开发示例和实验,以说明和理解如何通过各种javac编译器插件访问和操作javac AST。 该项目目前是以下子项目的容器: javac-ast-reader :这个 maven 项目提供了一个注释处理器的...

    词法分析[编译原理]

    词法分析是编译原理中的一个关键步骤,它是编译器设计的第一阶段,负责将源代码文本分解成一系列有意义的符号或标记,这些符号被称为“记号”(Token)。在这个过程中,词法分析器(也称为扫描器或 tokenizer)会...

    JAVAC源码 LR分析法 源代码

    总之,这份资源集合提供了深入理解`JAVAC`编译器和编译原理的宝贵资料,特别是对于那些对编译器设计感兴趣的开发者而言,通过对`JAVAC`源码的研究,可以学习到如何实现一个高效的词法分析器和LR分析器,这对于提升...

    Java编写的简单语法、词法分析

    例如,Java的`javac`编译器就是使用这些原理和工具构建的,它内部包含了词法分析和语法分析的实现,能将Java源代码转化为字节码。 学习编译原理不仅可以帮助你理解Java编译器如何工作,还能提升你在编程中的问题...

    javac源代码,java具体编译步骤

    这个过程涉及到许多编译阶段,包括词法分析、语法分析、语义分析以及优化等。以下是对`javac`源代码及Java编译步骤的详细解析。 1. **词法分析**:编译的第一步是读取源代码文件,并将其分解为一个个最小的有意义...

    javac--------src

    这个系列的内容可能涵盖了从源码级别的深入剖析javac的工作流程,包括词法分析、语法分析、语义分析、类文件生成等多个关键步骤。通过学习这些内容,开发者能够更深入地理解Java程序是如何从源代码转变为运行时的...

    JavaCC语法分析词法分析源代码

    JavaCC,全称为Java Compiler Compiler,是一款强大的工具,用于生成解析器和词法分析器,主要应用于处理基于Java语言的语法。它允许开发者定义自定义的语法,通过输入的源代码进行解析,进而构建出解释器或编译器。...

    词法检查工具

    3. 编译Java源代码:使用`javac`编译生成的Java文件,得到词法分析器类。 4. 运行词法分析器:将输入的源代码传递给生成的类,进行词法检查。 `jflex`不仅可以用于编译器和解释器的开发,还可以应用于任何需要文本...

    javac 编译器源代码

    1. **词法分析**:javac 首先读取源代码文件,将其分解为一个个称为“标记”(tokens)的最小单位,如关键字、标识符、操作符等。这个阶段主要由 `JavaTokenManager` 类负责。 2. **语法分析**:接着,javac 进行...

    Javac内核源码

    2. **语法分析**:词法分析后的标记被转换成抽象语法树(AST,Abstract Syntax Tree)结构,由 `com.sun.tools.javac.parser.Parser` 类实现。AST 是源代码的一种结构化表示,便于后续处理。 3. **类型检查**:在 `...

Global site tag (gtag.js) - Google Analytics