 Sam Jaffe
|
263f9ed594
Refactor tokenizer to use state machine/buffer.
|
4 rokov pred |
|
Sam Jaffe
|
d82dfc89f9
Adjust tests, better quote object
|
4 rokov pred |
|
Sam Jaffe
|
7eacca61a7
Add quote string option to tokenizer.
|
4 rokov pred |
|
Sam Jaffe
|
9b9852d627
Add option to escape dividers.
|
4 rokov pred |
|
Sam Jaffe
|
c6d0bc0584
Add coverage, and fill out tests.
|
5 rokov pred |
|
Sam Jaffe
|
ab55555d7f
Make bounds checking more elegant
|
5 rokov pred |
|
Sam Jaffe
|
cb0a1b37ac
Create the tokenize function in its entirety.
|
5 rokov pred |
