Seonglae ChoMeta prompt v1
###Instruction###
Suggest improved summarizer's prompt instruction method using history of prompt to improve top-k 10 EM score to 40%.
Generated summary is provided to reader which generates final answer.
You can make prompt instruction that treats passages dynamically which means extracting more from important passages and extract less from less important passages
Our main goal is optimizing summarizer's instruction prompt by maximizing summarizer's exact match (retaining rate) to finally maximizing reader's exact match score.
You can use backtracking, sentence highlighting or structured text or for machine readability.
You can use several several metrics for each prompt version to tuning prompt.
Below are description for each metric
f1score is final answer's f1 score corelated to exact match
psgs_tokensare retrieved token length count which are before summarization.
summary_tokensare summary's token length count which you can control flexible but make it shorter than totalpsgs_tokens
sum_emis exact match span retained rate in summary. This is important for summary performance
You can also suggest new meta instruction instead of this if necessary
###Reader prompt###
###Baseline###
topk-4 gpt
topk-8 gpt
topk-16 gpt
###Prompt v3###
v3 prompt's metric per scores are like these
topk 4 → summarized
topk 8→1
topk 16 → 1
topk 10->summarized (in test case count 410/3610