5 commits

Author	SHA1	Message	Date
janis	26c37e9dd3	test identifiers	2025-10-28 16:48:13 +01:00
janis	1886231dcd	return lexeme str slice from find_lexeme function	2025-10-28 16:30:15 +01:00
janis	cfc1c8e3b3	more tests for tokeniser	2025-10-28 13:12:27 +01:00
janis	63fbca378b	can tokenise simple tokens	2025-10-28 12:44:08 +01:00
janis	719451b935	reduce warnings, enable edition 2024, fix tokeniser (1/?)	2025-10-28 12:06:24 +01:00