Originally Posted by
gauvins
Interesting. I am considering Mistral and Llama 2 to run jobs I currently do with openai. Was there a reason behind your decision or was it just a matter of opportunity?
Have you compared with what Gemini/chatGPT/Claude have to say? Meaningful differences?
I haven't compared it with larger models that do better on the leader boards. This is mostly opportunity situation. I expect they may well do better on some of my tent questions.
It was also trying out a run local scenario. I was running locally on my laptop in a way I could also do without Internet access and without a GPU. So the model was smaller and less powerful but also more portable. The tool I was using was LM Studio but there are others like llamafile in similar space that I anticipate becoming more widespread on devices like mobile phones.
While not a surprise when you think about it, it was more intriguing to see both the presence and absence of spacial awareness. All the locations mentioned were at least in the general geographic area. The distances and overall order was bogus.
I see that as not too different from how people are exploring the ability to do math. That is an area where you need exact answers rather than something close (though looking at it in terms of fuzzier metric like how many digits are correct on a probability basis might also explain some of the claims of why some models appear to do much better). A similar approximation idea comes in my geographic questions.
Rather than go to larger models, the direction I am more interested to work towards is retrieval augmented generation (
https://arxiv.org/abs/2312.10997). In short what this does is augment a large language model with a content specific vector database that gets searched to provide context for the answer. So for example, if I could preload a set of trip reports about the C&O canal and then ask for a customized trip plan based on my circumstances or perhaps a set of blogs that describe equipment choices/results and have those incorporated in the answer.
I expect we could get better and more interesting answers that reflect previous documented trip reports. I also expect we can start to get tools that go through photos taken and summarize a trip. However a really good AI assistant for cycle touring will still take some work.