tree fd0b80ac48ad0514f02c7f541ba5004e719eb7fd
parent 1e5f93736445be0d4329122a93b90011993109f9
author Lalit Maganti <lalitm@google.com> 1726244846 +0100
committer Lalit Maganti <lalitm@google.com> 1726244846 +0100

tp: move tokenizer into its own folder

In prep for LALR based parsing of PerfettoSQL.

Change-Id: I7113adb1e8eda9b66d12cba95e222a4cdf91d814