Seonglae ChoReplaced before prompt
📈 METHOD PERFORMANCE: unified_method: Success rate: 7/7 Avg exact score: 0.107 Avg semantic score: 0.610 Avg processing time: 69.50s Avg matching score: 0.375 💰 Avg tokens per run: 250952 💰 Avg cost per run: $0.0404 💰 Total cost: $0.2825 🤖 Model: unknown original_method: Success rate: 7/7 Avg exact score: 0.087 Avg semantic score: 0.426 Avg processing time: 47.50s Avg matching score: 0.249 💰 Avg tokens per run: 212647 💰 Avg cost per run: $0.0346 💰 Total cost: $0.2419 🤖 Model: unknown hybrid_method: Success rate: 7/7 Avg exact score: 0.106 Avg semantic score: 0.646 Avg processing time: 114.85s 08 💰 Avg tokens per run: 213449 💰 Avg cost per run: $0.0358 💰 Total cost: $0.2505 🤖 Model: unknown clustering_method: Success rate: 7/7 Avg exact score: 0.143 Avg semantic score: 0.623 Avg processing time: 135.22s Avg matching score: 0.392 💰 Avg tokens per run: 231979 💰 Avg cost per run: $0.0420 💰 Total cost: $0.2940 🤖 Model: gpt-4o-mini direct_llm_method: Success rate: 7/7 Avg exact score: 0.085 Avg semantic score: 0.592 Avg processing time: 33.04s Avg matching score: 0.363 💰 Avg tokens per run: 80605 💰 Avg cost per run: $0.0127 💰 Total cost: $0.0889 🤖 Model: gpt-4o-mini pydantic_ai_method: Success rate: 7/7 Avg exact score: 0.162 Avg semantic score: 0.459 Avg processing time: 34.58s Avg matching score: 0.292 💰 Avg tokens per run: 82415 💰 Avg cost per run: $0.0133 💰 Total cost: $0.0928 🤖 Model: unknown sequential_pydantic: Success rate: 7/7 Avg exact score: 0.080 Avg semantic score: 0.316 Avg processing time: 140.56s Avg matching score: 0.209 💰 Avg tokens per run: 82568 💰 Avg cost per run: $0.0133 💰 Total cost: $0.0666 🤖 Model: unknown perfect_method: Success rate: 7/7 Avg exact score: 1.000 Avg semantic score: 1.000 Avg processing time: 0.00s Avg matching score: 1.000 💰 Avg tokens per run: 0 💰 Avg cost per run: $0.0000 💰 Total cost: $0.0000 🤖 Model: unknown dumb_method: Success rate: 7/7 Avg exact score: 0.000 Avg semantic score: 0.000 Avg processing time: 0.00s Avg matching score: 0.000 💰 Token usage: Not available 💾 CACHE PERFORMANCE: Total cache hits: 46 Total cache misses: 17 Cache hit rate: 73.0%
📈 METHOD PERFORMANCE: unified_method: Success rate: 7/7 Avg exact score: 0.122 Avg semantic score: 0.700 Avg processing time: 73.35s Avg matching score: 0.440 💰 Avg tokens per run: 318206 💰 Avg cost per run: $0.0508 💰 Total cost: $0.3558 🤖 Model: unknown original_method: Success rate: 7/7 Avg exact score: 0.106 Avg semantic score: 0.489 Avg processing time: 77.57s Avg matching score: 0.317 💰 Avg tokens per run: 335146 💰 Avg cost per run: $0.0555 💰 Total cost: $0.3883 🤖 Model: unknown hybrid_method: Success rate: 7/7 Avg exact score: 0.093 Avg semantic score: 0.720 Avg processing time: 528.95s Avg matching score: 0.445 💰 Avg tokens per run: 335928 💰 Avg cost per run: $0.0573 💰 Total cost: $0.4010 🤖 Model: unknown clustering_method: Success rate: 7/7 Avg exact score: 0.122 Avg semantic score: 0.585 Avg processing time: 209.73s Avg matching score: 0.341 💰 Avg tokens per run: 360825 💰 Avg cost per run: $0.0651 💰 Total cost: $0.4556 🤖 Model: gpt-4o-mini direct_llm_method: Success rate: 7/7 Avg exact score: 0.143 Avg semantic score: 0.767 Avg processing time: 111.73s Avg matching score: 0.579 💰 Avg tokens per run: 81373 💰 Avg cost per run: $0.0131 💰 Total cost: $0.0916 🤖 Model: gpt-4o-mini pydantic_ai_method: Success rate: 7/7 Avg exact score: 0.075 Avg semantic score: 0.385 Avg processing time: 208.52s Avg matching score: 0.231 💰 Avg tokens per run: 94392 💰 Avg cost per run: $0.0182 💰 Total cost: $0.0908 🤖 Model: unknown sequential_pydantic: Success rate: 7/7 Avg exact score: 0.126 Avg semantic score: 0.593 Avg processing time: 58.05s Avg matching score: 0.423 💰 Avg tokens per run: 84436 💰 Avg cost per run: $0.0139 💰 Total cost: $0.0976 🤖 Model: unknown perfect_method: Success rate: 7/7 Avg exact score: 1.000 Avg semantic score: 1.000 Avg processing time: 0.00s Avg matching score: 1.000 💰 Avg tokens per run: 0 💰 Avg cost per run: $0.0000 💰 Total cost: $0.0000 🤖 Model: unknown dumb_method: Success rate: 7/7 Avg exact score: 0.000 Avg semantic score: 0.000 Avg processing time: 0.00s Avg matching score: 0.000 💰 Token usage: Not available 💾 CACHE PERFORMANCE: Total cache hits: 0 Total cache misses: 63 Cache hit rate: 0.0%
Trimmed before/after prompt
📈 METHOD PERFORMANCE: unified_method: Success rate: 7/7 Avg exact score: 0.069 Avg semantic score: 0.422 Avg processing time: 25.94s Avg matching score: 0.235 💰 Avg tokens per run: 410098 💰 Avg cost per run: $0.0636 💰 Total cost: $0.4451 🤖 Model: unknown original_method: Success rate: 7/7 Avg exact score: 0.069 Avg semantic score: 0.375 Avg processing time: 39.26s Avg matching score: 0.200 💰 Avg tokens per run: 422211 💰 Avg cost per run: $0.0666 💰 Total cost: $0.4661 🤖 Model: unknown hybrid_method: Success rate: 7/7 Avg exact score: 0.089 Avg semantic score: 0.566 Avg processing time: 159.75s Avg matching score: 0.339 💰 Avg tokens per run: 425239 💰 Avg cost per run: $0.0693 💰 Total cost: $0.4849 🤖 Model: unknown clustering_method: Success rate: 7/7 Avg exact score: 0.092 Avg semantic score: 0.551 Avg processing time: 607.81s Avg matching score: 0.322 💰 Avg tokens per run: 454201 💰 Avg cost per run: $0.0789 💰 Total cost: $0.5523 🤖 Model: gpt-4o-mini direct_llm_method: Success rate: 7/7 Avg exact score: 0.099 Avg semantic score: 0.474 Avg processing time: 31.50s Avg matching score: 0.244 💰 Avg tokens per run: 103817 💰 Avg cost per run: $0.0161 💰 Total cost: $0.1126 🤖 Model: gpt-4o-mini pydantic_ai_method: Success rate: 7/7 Avg exact score: 0.077 Avg semantic score: 0.373 Avg processing time: 29.15s Avg matching score: 0.185 💰 Avg tokens per run: 105100 💰 Avg cost per run: $0.0164 💰 Total cost: $0.1150 🤖 Model: unknown sequential_pydantic: Success rate: 7/7 Avg exact score: 0.058 Avg semantic score: 0.346 Avg processing time: 23.68s Avg matching score: 0.168 💰 Avg tokens per run: 104385 💰 Avg cost per run: $0.0161 💰 Total cost: $0.1127 🤖 Model: unknown perfect_method: Success rate: 7/7 Avg exact score: 1.000 Avg semantic score: 1.000 Avg processing time: 0.00s Avg matching score: 1.000 💰 Avg tokens per run: 0 💰 Avg cost per run: $0.0000 💰 Total cost: $0.0000 🤖 Model: unknown dumb_method: Success rate: 7/7 Avg exact score: 0.000 Avg semantic score: 0.000 Avg processing time: 0.00s Avg matching score: 0.000 💰 Token usage: Not available 💾 CACHE PERFORMANCE: Total cache hits: 63 Total cache misses: 0 Cache hit rate: 100.0%
📈 METHOD PERFORMANCE: unified_method: Success rate: 7/7 Avg exact score: 0.057 Avg semantic score: 0.531 Avg processing time: 65.45s Avg matching score: 0.321 💰 Avg tokens per run: 411793 💰 Avg cost per run: $0.0644 💰 Total cost: $0.4505 🤖 Model: unknown original_method: Success rate: 7/7 Avg exact score: 0.050 Avg semantic score: 0.351 Avg processing time: 56.29s Avg matching score: 0.196 💰 Avg tokens per run: 422546 💰 Avg cost per run: $0.0665 💰 Total cost: $0.4658 🤖 Model: unknown hybrid_method: Success rate: 7/7 Avg exact score: 0.097 Avg semantic score: 0.635 Avg processing time: 124.07s Avg matching score: 0.412 💰 Avg tokens per run: 427415 💰 Avg cost per run: $0.0701 💰 Total cost: $0.4908 🤖 Model: unknown clustering_method: Success rate: 7/7 Avg exact score: 0.083 Avg semantic score: 0.526 Avg processing time: 194.26s Avg matching score: 0.313 💰 Avg tokens per run: 459523 💰 Avg cost per run: $0.0809 💰 Total cost: $0.5664 🤖 Model: gpt-4o-mini direct_llm_method: Success rate: 7/7 Avg exact score: 0.076 Avg semantic score: 0.587 Avg processing time: 459.17s Avg matching score: 0.408 💰 Avg tokens per run: 104811 💰 Avg cost per run: $0.0166 💰 Total cost: $0.1163 🤖 Model: gpt-4o-mini pydantic_ai_method: Success rate: 7/7 Avg exact score: 0.064 Avg semantic score: 0.299 Avg processing time: 112.01s Avg matching score: 0.146 💰 Avg tokens per run: 106761 💰 Avg cost per run: $0.0168 💰 Total cost: $0.1010 🤖 Model: unknown sequential_pydantic: Success rate: 7/7 Avg exact score: 0.067 Avg semantic score: 0.380 Avg processing time: 27.02s Avg matching score: 0.200 💰 Avg tokens per run: 105026 💰 Avg cost per run: $0.0163 💰 Total cost: $0.1142 🤖 Model: unknown perfect_method: Success rate: 7/7 Avg exact score: 1.000 Avg semantic score: 1.000 Avg processing time: 0.00s Avg matching score: 1.000 💰 Avg tokens per run: 0 💰 Avg cost per run: $0.0000 💰 Total cost: $0.0000 🤖 Model: unknown dumb_method: Success rate: 7/7 Avg exact score: 0.000 Avg semantic score: 0.000 Avg processing time: 0.00s Avg matching score: 0.000 💰 Token usage: Not available 💾 CACHE PERFORMANCE: Total cache hits: 0 Total cache misses: 63 Cache hit rate: 0.0%