Seonglae ChoThe completions API endpoint received its final update in July 2023 and has a different interface than the new chat completions endpoint. Instead of the input being a list of messages, the input is a freeform text string called a
prompt.JSON mode will not guarantee the output matches any specific schema, only that it is valid and parses without errors.
accounts for 97% of API GPT usage
Chat Completions API Notion
Log probability as argument
OpenAI Platform
Explore developer resources, tutorials, API docs, and dynamic examples to get the most out of OpenAI's platform.
https://platform.openai.com/docs/api-reference/chat
Predicted output
Any tokens provided in predictions that are not part of the final completion will be charged at completion token rates. This means the more an output differs from the prediction, the more it will cost. Using Predicted Outputs can reduce processing time.
OpenAI Platform
Explore developer resources, tutorials, API docs, and dynamic examples to get the most out of OpenAI's platform.
https://platform.openai.com/docs/guides/latency-optimization#use-predicted-outputs
Models
OpenAI Platform
Explore developer resources, tutorials, API docs, and dynamic examples to get the most out of OpenAI's platform.
https://platform.openai.com/docs/models/overview
Prompt Caching for cost reduction and Latency impprovement (Automatic Application)
Prompt Caching in the API
Offering automatic discounts on inputs that the model has recently seen
https://openai.com/index/api-prompt-caching/
OpenAI Prompt Caching in GPT 4o and o1: How Does It Compare To Claude Prompt Caching? - Bind AI
OpenAI recently introduced prompt caching features as a part of its annual DevDay announcements. Prompt caching—which OpenAI claims can benefit users with a 50% discount on inputs—will now applied to various models, including GPT-4o and its mini versions. Unsurprisingly, this has generated excitement among developers, with many already drawing comparisons between OpenAI's and Claude's prompt
https://blog.getbind.co/2024/10/03/openai-prompt-caching-how-does-it-compare-to-claude-prompt-caching/
