doctrine/lexer

★ 11,166PHPAudience · developerComplexity · 2/5Setup · moderate

Mindmap

mindmap
  root((repo))
    What it does
      Base lexer class
      Token reading
      Parser foundation
    How it works
      Extend base class
      Define token types
      Recursive descent
    Used by
      Doctrine ORM DQL
      Doctrine Annotations
    Audience
      PHP developers
      Parser builders

mindmap root((repo)) What it does Base lexer class Token reading Parser foundation How it works Extend base class Define token types Recursive descent Used by Doctrine ORM DQL Doctrine Annotations Audience PHP developers Parser builders

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Extend the base lexer class to tokenize a custom query language or configuration file format in PHP.

USE CASE 2

Use it as a foundation for a recursive descent parser that processes structured text input.

USE CASE 3

Add Doctrine ORM to a PHP project and benefit from this library's tokenization indirectly for Doctrine Query Language.

Tech stack

PHP

Getting it running

Difficulty · moderate Time to first run · 1h+

Documentation is minimal in the README, study Doctrine ORM or Annotations source code for real usage examples.

In plain English

Doctrine Lexer is a PHP library that provides a base class for building a lexer. A lexer is a program component that reads raw text and breaks it into a sequence of labeled tokens, which is the first step in parsing a custom language or query syntax. For example, if you had a query like "SELECT name FROM users WHERE age > 30", a lexer would identify each word and symbol as a specific type of token before any other logic tries to understand what the query means. This library is described as a base for use in top-down, recursive descent parsers. That refers to a common style of parser design where the code processes tokens from left to right and builds up an understanding of the input by calling itself recursively. The library does not implement a full parser but gives you the foundational tools to build one. The README is short and says this lexer is used internally by two other Doctrine projects: Doctrine Annotations (a PHP annotation parsing library) and Doctrine ORM, specifically for its Doctrine Query Language (DQL). Doctrine is a well-known set of PHP database and object persistence libraries. The high star count on this repository reflects how many PHP projects depend on Doctrine ORM and therefore pull in this library indirectly. There is no detailed usage documentation in the README itself. Developers who need to use or extend this library are pointed to the project page at doctrine-project.org. Given how brief the repository description is, it is clearly intended for developers already working within the Doctrine ecosystem rather than newcomers. The high star count reflects indirect popularity, since many PHP applications that use Doctrine ORM automatically pull in this library as a transitive dependency.

Copy-paste prompts

Prompt 1

I'm building a PHP parser for a custom query syntax. How do I extend Doctrine Lexer's base class to tokenize expressions like [example syntax]?

Prompt 2

Show me how to define token types and patterns in Doctrine Lexer so I can parse [describe format].

Prompt 3

Write a recursive descent parser in PHP using Doctrine Lexer that handles simple arithmetic like 2 + 3 * 4.

Prompt 4

Explain how Doctrine ORM uses doctrine/lexer to parse DQL queries, and how I can use the same pattern for my own query language.

Open on GitHub → Explain another repo

← doctrine on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.