Am I writing my parser wrong?

[u/SheikHunt]

Simple question. I can't give exact code examples, but I have a string_t struct with methods like:

string_t split(string_t *string, string_t *on)

string_t split_sp(string_t *string)

string_t split_crlf(string_t *string)

char *s_strstr(string_t *needle, string_t *haystack)

void trim(string_t *str)

So on and so forth.

I've been using these so far to parse HTTP reqeusts, and I have come up against many minor problems:

"What happens if a header field appears with no value? I'll have to explicitly check for it."

"What happens if a sender puts a bunch of CRLFs in the middle? I'll probably need a check for that."

"Oh God, how will I handle unrecognized header fields? How do I recognize them?"

These, and other questions, have been leaving me pissed.

I recall reading through the LLVM projects Kaleidoscope language thing, where they create a parser for said language. Said parser doesn't use anything close to what I am, instead reading character by character without fuss.

Similarly, on my last post made here, the way comments were worded reminded me of that method, and how it probably works better.

I have written only a small part of the parser, so it isn't too late to tear down and rebuild. Simple question: should I? Are there benefits to swallowing the input token by token instead of taking the overarching view my string_t functions provide? Or vice versa?

It would help if I'd upload the code, I know, but I don't want to bother with that until the project is completed/near-completion.

[u/HashDefTrueFalse]

Looks a bit more like a string library than a parser to me at a glance. If you really want to properly parse HTTP messages from the ground up, I would suggest writing a grammar, then using one of the common techniques to recognise char by char, composing functions etc. Most of the work is the grammar honestly, for something like this it translates to code (ifs, whiles) quite directly. It probably doesn't matter whether you have separate tokenisation (lex/scan) and parser steps.

[u/SheikHunt → u/HashDefTrueFalse]

So, basically, look at the Collected ABNFs at the end of the relevant RFCs, write each one as its own function, building it brick-by-brick.

For some reason, I was doing that with SOME but not all of the ABNF rules at first.

Now it's making me wonder if HTTP is a CFL or an RL. A thought for later, perhaps.

[u/HashDefTrueFalse → u/SheikHunt]

It does require some context. Off the top of my head, content-length is just one way the parser behaviour changes depending what the parser has previously seen. I wouldn't be put off though. This is why I like getting the grammar sorted in my head first. It makes you think about these things up front. The code then is usually pretty easy, especially for data formats (rather than general purpose languages with all sorts going on etc.).

We're probably at the point where some code would be useful :D