Have you developed any applications (web or mobile) incorporating chatgpt? Can you share any tips on how to go about doing it, what kind of applications are suitable for it? Or do you know any published guides on an approach?
Have you developed any applications (web or mobile) incorporating chatgpt? Can you share any tips on how to go about doing it, what kind of applications are suitable for it? Or do you know any published guides on an approach?
This is still such a new field that high quality writing about it is still hard to come by.
This prompt engineering guide from Brex is very good: https://github.com/brexhq/prompt-engineering
Here are some notes I’ve made on things I’ve built so far:
I skimmed over these and interestingly enough, I couldn’t find any mention of optimizing API costs. (i.e. minimizing input and output tokens). In my own prompting journey, I found API costs (especially for gpt4) to be the biggest obstacle. I feel that many of LLM’s inaccuracies and hallucinations can be solved by techniques like adding guard-rails in the prompt, self-inspection, or averaging repeated queries, but they all inflate the measure of “token per new information”, which can get costly for large domains. And this is not a problem that’s entirely without solution. I was able to improve it by preprocessing the text, packing queries, even just asking chatgpt to output CSV instead of JSON. (another thing I surprisingly couldn’t find any mention of)
The cheesiest I/O optimisation I’ve found is starting or ending a request with “be concise”. OpenAI’s GPT will still usually reply with “Sure,” but it cuts output dramatically:
versus:
EDIT: versus:
I suggested a change to the heading to clarify that you mean developing apps that use ChatGPT, rather than using ChatGPT to develop apps. I suspect a lot of people are skipping this because they think the question is the latter.
Appreciate the change David. Thanks
In addition to the useful links @simonw posted, here are a few more:
One thing to note is that at first you’ll be limited to using the gpt-3.5-turbo model in the API, since gpt-4 API access is waitlisted. That being said, the waitlist isn’t that long — it took about a month for me. It seems like OpenAI prioritizes people who use the 3.5 API; my hunch is they let you into 4 not long after you pay your first bill from using 3.5. Try just using 3.5 first, but if it isn’t working well even after trying various techniques e.g. CoT prompting from those guides above, don’t despair — 4 is actually a lot better at following complex prompts.
Or you can try the open source LLMs, but be forewarned that the licensing for the model weights (aka, anything useful) forbids commercial use.
I’ve made a command line tool that tries to come up with a unix command line based on a natural language description. https://github.com/rain-1/gpt.sh
I’ve made a modification to an IRC bot so that you are able to invoke commands using natural language. https://github.com/rain-1/sopel/tree/gpt-branch