LLMs are designed to hallucinate
Hallucination in the LLM world simply means the model made a bad prediction. The model generates a token, or series of tokens that doesn't have any real world viability or doesn't relate to your original input. I'd also consider any result that explicitly against your instructions a form a hallucination as well.
One example I came across recently was when I was asking GPT-4o to give me some information around a ski resort such as address and website and the model returned the incorrect website url. The website looked like it could be the correct url, but it wasn't.
As a human, we may read the entire website as a single entity such as "https://www.supercoolwebsite.com" but when the LLM sees a a bunch of different tokens.
OpenAI splits "https://www.supercoolwebsite.com" into 7 different tokens:
https://www.supercoolwebsite.com
Which means there are 7 chances for the wrong prediction to happen, especially if one of these tokens, or series of tokens, doesn't have any strong associations to the factual information or original query.
Every prompt you are sending the LLM has to generate something, correct, or incorrect it doesn't matter. Thats the models entire job, just to predict the next token.
How to avoid LLM hallucinations? One technique I've been using is to give the model an "out", meaning give an option on what to return. If my prompt has a very broad set of instructions, giving the options to return something else seems to help. For example, if I'm asking to "fix this code" but there is nothing to fix, adding an additional instruction of "if there is nothing to fix, return the text NOFIX". Of course, your mileage may vary depending on the context and task.