Seonglae Chov4
The passages provided are related to the question. Make one or two bullet points given passages to be easier for the reader answering the given question. Make bullet points for each passages information that can be an evidence to answer the question. Find the evidences that support the an swer to the question and retain them. The rewrited text should be shorter than the total length of the original passages. Use only information in the document. Print only the rewrited texts Reduce the noise unrelated to answer the question. Remove unrelated phrases, sentences and information to answer the question. The final answer for this question is contained is the passages so maintain the exact span of answer smaller than 5 words. Maintain exact detailed information, not abstract them.
[3000:3200] {'exact_match': 36.0, 'f1': 46.63733766233766, 'psgs_tokens': 1372.265, 'summary_tokens': 59.77, 'sum_tn': 1.0, 'sum_fn': 18.5, 'ret_em': 73.0, 'sum_em': 55.50000000000001, 'read_fp': 20.0, 'read_tn': 0.5}
첫결과 성능 좋다 summary 가 아주 짧게 나와서 psgs 구분 명확히 해줘야할듯
짧은 토큰과 summarizer exact match가 상대적으로 낮은 것에 비해 성능이 높게 나온 것으로 보아 machine readability가 상승한 것으로 볼 수 있다
다만 200개라서 좀 튜닝하고 숫자 늘려봐야
The passages provided are related to the question. Make one or two bullet points per given passage to be easier for the reader answering the given question. Make bullet points for each passages information that can be an evidence to answer the question. Find the evidences that support the an swer to the question and retain them. The rewrited text should be shorter than the total length of the original passages. Use only information in the document. Print only the rewrited texts by concatenating all bullet points. Reduce the noise unrelated to answer the question. Remove unrelated phrases, sentences and information to answer the question. The final answer for this question is contained is the passages so maintain the exact span of answer smaller than 5 words. Maintain exact detailed information, not abstract them.
{'exact_match': 33.0, 'f1': 44.49642857142857, 'psgs_tokens': 1372.515, 'summary_tokens': 110.745, 'sum_tn': 0.0, 'sum_fn': 15.5, 'ret_em': 73.5, 'sum_em': 57.99999999999999, 'read_fp': 25.0, 'read_tn': 0.0}
같은 구간인데 토큰은 늘고 성능은 줄었다
sum_em은 유사한 걸 보니 bullet point로 얻을 수 있는 한계가 있을수도
The passages provided are related to the question. Make one or two bullet points per given passages to be easier for the reader answering the given question. Make bullet points for each passages information that can be an evidence to answer the question. Find the evidences that support the answer to the question and retain them. The rewrited text should be shorter than the total length of the original passages. Use only information in the document. Print only the rewrited texts Reduce the noise unrelated to answer the question. Remove unrelated phrases, sentences and information to answer the question. The final answer for this question is contained is the passages so maintain the exact span of answer smaller than 5 words. Maintain exact detailed information, not abstract them.
{'exact_match': 32.5, 'f1': 42.282251082251086, 'psgs_tokens': 1372.15, 'summary_tokens': 54.44, 'sum_tn': 0.0, 'sum_fn': 17.0, 'ret_em': 73.5, 'sum_em': 56.49999999999999, 'read_fp': 24.5, 'read_tn': 0.5}
점점 내린다 뭐지
전체 v4 해보니 스코어가 안나온다 쩝
{'exact_match': 32.96398891966759, 'f1': 43.55914294264897, 'psgs_tokens': 1368.1986149584488, 'summary_tokens': 57.50221606648199, 'sum_tn': 0.2493074792243767, 'sum_fn': 18.587257617728532, 'ret_em': 75.4016620498615, 'sum_em': 57.06371191135734, 'read_fp': 24.570637119113574, 'read_tn': 0.47091412742382266}