Total Calls
70
2 days of testing
Avg Rating
3.41
34 🟢
27 🟡
8 🔴
Deflection Rate
5.7%
51.4% transferred
Personalization
68.6%
+40.6% vs Day 1
Total Cost
$8.05
$0.11 avg/call
Avg Duration
47s
KB search 55.7% of calls
Day:
Quality:
Pattern:
70 calls
Day-over-Day Comparison
Day 1
38 calls3.41
Avg Rating
2.6%
Deflection
$4.52
Cost
🟢 48.6%
🟡 40.5%
🔴 10.8%
👤 50.0%
Day 2
32 calls3.41
Avg Rating
9.4%
Deflection
$3.53
Cost
🟢 50.0%
🟡 37.5%
🔴 12.5%
👤 90.6%
Rating Distribution
Click a segment to filter
Call Patterns (outcome)
Click a bar to filter
Top Caller Intents (stated purpose)
Click a bar to search
Call Log
Click row to expand
| Day | # | Call ID | Time | Dur | Cost | Rating | Pattern | Intent | Personalized | Flags |
|---|---|---|---|---|---|---|---|---|---|---|
| Day 1 | 1 | 019cfd96-7f3... | 4:57 PM | 46s | $0.12 | 3/5 | KB+Transfer | Remove a mailbox | Esther | 🔴 |
|
First User Message
"Remove a mailbox."
First Bot Response
"The search results provided information on deleting a duplicate mail item..."
Notes
Bot exposed internal search logic.
Details
customer-ended-call
V1
search_knowledge_base
🔴 Exposed internal search logic
|
||||||||||
| Day 1 | 2 | 019cfd95-a50... | 4:56 PM | 44s | $0.11 | 4/5 | KB+Transfer | Rejected mailbox address | — | |
|
First User Message
"Rejected mailbox address."
First Bot Response
"If your address was rejected, it might be due to an error..."
Notes
Reasonable answer. Clean transfer.
Details
customer-ended-call
V1
search_knowledge_base
|
||||||||||
| Day 1 | 3 | 019cfd92-7b7... | 4:52 PM | 109s | $0.17 | 2/5 | KB+Transfer | Mailbox setup / getting started | — | 🔴 |
|
First User Message
"Mail mailbox."
First Bot Response
"To set up your mailbox, you need a 1583 form, IDs, and notarization."
Notes
Silence-timed-out. Caller may have been overwhelmed.
Details
silence-timed-out
V1
search_knowledge_base
🔴 Silence timeout
|
||||||||||
| Day 1 | 4 | 019cfd88-f9d... | 4:42 PM | 43s | $0.11 | 2/5 | KB+Transfer | Received someone else's mail | — | 🔴 |
|
First User Message
"Somebody else's mail was in my mailbox."
First Bot Response
"If you've received mail not addressed to you, it may have been placed by mistake."
Notes
Caller said answer was wrong.
Details
customer-ended-call
V1
search_knowledge_base
🔴 Caller said answer was wrong
|
||||||||||
| Day 1 | 5 | 019cfd88-441... | 4:41 PM | 39s | $0.13 | 3/5 | KB+Transfer | Cancel subscription | — | 🟡 |
|
First User Message
"I want to cancel my subscription."
First Bot Response
"To cancel, log in and go to mailbox settings..."
Notes
Caller wanted agent to cancel.
Details
customer-ended-call
V1
search_knowledge_base
🟡 KB gap: Cancellation
|
||||||||||
| Day 1 | 6 | 019cfd86-e1a... | 4:40 PM | 60s | $0.15 | 3/5 | KB+Transfer | Update debit card number | Michael | 🟡 |
|
First User Message
"I need to update my debit card number."
First Bot Response
"To update your card, go to account settings."
Notes
Premature sign-off — bot said goodbye too early.
Details
customer-ended-call
V1
search_knowledge_base
🟡 Premature sign-off
|
||||||||||
| Day 1 | 7 | 019cfd86-c27... | 4:39 PM | 21s | $0.05 | 4/5 | Immediate Transfer | Immediate agent transfer | — | |
|
First User Message
"Speak to operator."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean transfer.
Details
customer-ended-call
V1
|
||||||||||
| Day 1 | 8 | 019cfd83-922... | 4:36 PM | 10s | $0.03 | — | Immediate Transfer | Unknown — hung up during greeting | — | |
|
First User Message
"N/A"
First Bot Response
"N/A"
Notes
Hung up before bot finished greeting.
Details
customer-ended-call
V1
|
||||||||||
| Day 1 | 9 | 019cfd83-4f7... | 4:36 PM | 30s | $0.09 | 3/5 | KB+Transfer | Cancel subscription / remove payment | Sterling | 🟡 |
|
First User Message
"I need to terminate my account and remove my payment."
First Bot Response
"I don't have specific details on terminating accounts."
Notes
KB gap — cancellation.
Details
customer-ended-call
V1
search_knowledge_base
🟡 KB gap: Cancellation
|
||||||||||
| Day 1 | 10 | 019cfd7e-2cf... | 4:30 PM | 66s | $0.18 | 4/5 | KB+Transfer | Change mail center location | Nyla | |
|
First User Message
"How do I sign up for a different location?"
First Bot Response
"Could you clarify — accessing mailbox or changing location?"
Notes
Good clarifying question, then accurate KB answer.
Details
customer-ended-call
V1
search_knowledge_base
|
||||||||||
| Day 1 | 11 | 019cfd7d-f96... | 4:30 PM | 22s | $0.05 | 4/5 | Immediate Transfer | Immediate agent transfer | — | |
|
First User Message
"Transfer me to a live agent."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean transfer.
Details
customer-ended-call
V1
|
||||||||||
| Day 1 | 12 | 019cfd79-526... | 4:25 PM | 27s | $0.08 | 3/5 | KB+Transfer | Billed twice for same mailing | Barbara | |
|
First User Message
"I was billed twice for the same mailing."
First Bot Response
"I don't have specific information on billing disputes."
Notes
Account-specific billing issue.
Details
customer-ended-call
V1
search_knowledge_base
|
||||||||||
| Day 1 | 13 | 019cfd77-ba5... | 4:23 PM | 41s | $0.10 | 3/5 | KB+Transfer | Requirements for Canadian customers | — | 🟡 |
|
First User Message
"What are the requirements for customers in Canada?"
First Bot Response
"Requirements may vary by location..."
Notes
Vague answer — KB should have info.
Details
customer-ended-call
V1
search_knowledge_base
🟡 Vague answer
|
||||||||||
| Day 1 | 14 | 019cfd73-644... | 4:18 PM | 20s | $0.05 | 4/5 | Immediate Transfer | Immediate agent transfer | Tyra | |
|
First User Message
"Agent."
First Bot Response
"Let me connect you with a support agent now."
Notes
Instant transfer.
Details
customer-ended-call
V1
|
||||||||||
| Day 1 | 15 | 019cfd73-5e0... | 4:18 PM | 48s | $0.12 | 3/5 | KB+Transfer | Copy of lease agreement | — | 🟡 |
|
First User Message
"I need a copy of the lease agreement for my box."
First Bot Response
"I don't have specific information about lease agreements."
Notes
Same KB gap as Call 17.
Details
customer-ended-call
V1
search_knowledge_base
🟡 KB gap: Lease agreements
|
||||||||||
| Day 1 | 16 | 019cfd71-6aa... | 4:16 PM | 23s | $0.06 | 4/5 | Immediate Transfer | Doesn't like AI, wants agent | Elle | |
|
First User Message
"Can you get me a live person? I don't like speaking to AI."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean transfer.
Details
customer-ended-call
V1
|
||||||||||
| Day 1 | 17 | 019cfd6b-fe9... | 4:10 PM | 44s | $0.12 | 3/5 | KB+Transfer | Lease agreement for account | Shauna | 🟡 |
|
First User Message
"My account includes a lease agreement."
First Bot Response
"I don't have specific information about lease agreements."
Notes
KB gap — lease agreements.
Details
customer-ended-call
V1
search_knowledge_base
🟡 KB gap: Lease agreements
|
||||||||||
| Day 1 | 18 | 019cfd6a-24a... | 4:08 PM | 80s | $0.19 | 4/5 | KB+Transfer | Can't update payment card (Venmo) | Marcus | |
|
First User Message
"I've been trying to update my card but it won't take my Venmo card."
First Bot Response
"To update your payment method, go to Account Settings..."
Notes
Good detailed answer.
Details
customer-ended-call
V1
search_knowledge_base
|
||||||||||
| Day 1 | 19 | 019cfd69-de2... | 4:08 PM | 53s | $0.17 | 3/5 | KB+Transfer | Can't access account for 3 days | Dennis | 🔴 |
|
First User Message
"It's three days I'm unable to get access to my account."
First Bot Response
"To schedule a mail pickup, log in to your account..."
Notes
Tone-deaf — told caller who can't log in to log in.
Details
customer-ended-call
V1
search_knowledge_base
🔴 Tone-deaf response
|
||||||||||
| Day 1 | 20 | 019cfd68-290... | 4:06 PM | 80s | $0.17 | 4/5 | KB+Transfer | App not working on phone | — | |
|
First User Message
"How can I fix the iPostal1 app?"
First Bot Response
"Try checking internet, updating the app, logging out and back in..."
Notes
Solid troubleshooting answer.
Details
customer-ended-call
V1
search_knowledge_base
|
||||||||||
| Day 1 | 21 | 019cfd65-43f... | 4:03 PM | 105s | $0.14 | 2/5 | Immediate Transfer | Unknown — confused | — | 🔴 |
|
First User Message
"What did you say?"
First Bot Response
"I can help with questions about your iPostal1 virtual mailbox..."
Notes
Silence-timed-out. Caller confused by greeting.
Details
silence-timed-out
V1
🔴 Silence timeout — caller abandoned
|
||||||||||
| Day 1 | 22 | 019cfd64-96e... | 4:02 PM | 138s | $0.20 | 2/5 | DTMF | App not working + DTMF 0 | — | 🔴 |
|
First User Message
"My iPostal1 app is not working."
First Bot Response
"Let me transfer you to a live agent..."
Notes
DTMF 0 pressed. Transfer may have failed — silence timeout.
Details
silence-timed-out
V1
🔴 Possible failed transfer — silence timeout
|
||||||||||
| Day 1 | 23 | 019cfd64-3b2... | 4:02 PM | 62s | $0.17 | 5/5 | KB+Transfer | Refund for storage fees on picked-up mail | — | |
|
First User Message
"I need a refund for an item I already picked up."
First Bot Response
"For a refund on storage fees, you'll need mail center approval..."
Notes
Excellent response. Best call of session.
Details
customer-ended-call
V1
search_knowledge_base
|
||||||||||
| Day 1 | 24 | 019cfd62-d8b... | 4:00 PM | 42s | $0.11 | 3/5 | Immediate Transfer | Confused/skeptical about AI | — | |
|
First User Message
"What did you say?"
First Bot Response
"I can help with questions about your iPostal1 virtual mailbox..."
Notes
Caller confused about AI. Bot transferred.
Details
customer-ended-call
V1
|
||||||||||
| Day 1 | 25 | 019cfd62-74c... | 4:00 PM | 23s | $0.05 | 4/5 | Immediate Transfer | Immediate agent transfer | — | |
|
First User Message
"Can I speak to an operator, please?"
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean transfer.
Details
customer-ended-call
V1
|
||||||||||
| Day 1 | 26 | 019cfd61-a2f... | 3:59 PM | 21s | $0.06 | 4/5 | Immediate Transfer | Immediate agent transfer (frustrated) | — | |
|
First User Message
"I wanted to speak to an agent."
First Bot Response
"Let me connect you with a support agent now."
Notes
Bot transferred immediately.
Details
customer-ended-call
V1
|
||||||||||
| Day 1 | 27 | 019cfd60-f68... | 3:58 PM | 58s | $0.14 | 4/5 | KB+Transfer | Change mailbox location | Chuck | |
|
First User Message
"I would like to change my location on my mailbox."
First Bot Response
"You'll need a new 1583 form for a location change."
Notes
Good KB answer with accurate process info.
Details
customer-ended-call
V1
search_knowledge_base
|
||||||||||
| Day 1 | 28 | 019cfd60-d66... | 3:58 PM | 60s | $0.17 | 3/5 | KB+Transfer | Payment declining | Emmanuel | |
|
First User Message
"My payments have been declining."
First Bot Response
"You may need to update your payment method..."
Notes
Generic payment advice. Correctly escalated.
Details
customer-ended-call
V1
search_knowledge_base
|
||||||||||
| Day 1 | 29 | 019cfd60-c76... | 3:58 PM | 69s | $0.13 | 4/5 | Deflected | How to get my mail / schedule pickup | Shannon | |
|
First User Message
"How can I get my mail?"
First Bot Response
"To schedule a pickup, log in and go to your mailbox..."
Notes
Successful deflection. Caller appeared satisfied.
Details
customer-ended-call
V1
search_knowledge_base
|
||||||||||
| Day 1 | 30 | 019cfd5f-be8... | 3:57 PM | 68s | $0.17 | 4/5 | KB+Transfer | Change business name on account | — | |
|
First User Message
"I would like to change the business name on my account."
First Bot Response
"To change your business name, contact support. New verification required."
Notes
Good KB answer.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
|
||||||||||
| Day 1 | 31 | 019cfd5e-2e5... | 3:55 PM | 31s | $0.09 | 3/5 | Immediate Transfer | Password reset / can't log in | — | 🟡 |
|
First User Message
"I need help getting in my iPostal. It won't send me a link to reset my password."
First Bot Response
"I recommend connecting you with a live agent."
Notes
Bot went straight to transfer without KB search.
Details
customer-ended-call
V1
transfer_to_agent
🟡 KB search not attempted
|
||||||||||
| Day 1 | 32 | 019cfd5d-62b... | 3:54 PM | 88s | $0.19 | 4/5 | KB+Transfer | International customer — requirements + 1583 status | Jeff | |
|
First User Message
"I'm an international customer, what requirements are there?"
First Bot Response
"For international customers, you can use online notarization via proof.com..."
Notes
Excellent KB answer. 1583 tool invoked but failed — handled gracefully.
Details
customer-ended-call
V1
search_knowledge_base
check_1583_status
transfer_to_agent
|
||||||||||
| Day 1 | 33 | 019cfd5b-a2f... | 3:52 PM | 21s | $0.06 | 4/5 | Immediate Transfer | Immediate agent transfer | Jeff | |
|
First User Message
"Agent please."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean transfer.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 1 | 34 | 019cfd5b-89b... | 3:52 PM | 53s | $0.14 | 4/5 | KB+Transfer | Mail center operator — client has different company name | — | |
|
First User Message
"I need to speak about a client."
First Bot Response
"Could you please provide more details..."
Notes
Bot correctly identified account-specific issue.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
|
||||||||||
| Day 1 | 35 | 019cfd5b-175... | 3:52 PM | 10s | $0.03 | 3/5 | Immediate Transfer | Unknown — hung up during greeting | Jeff | |
|
First User Message
"N/A"
First Bot Response
"N/A"
Notes
Caller hung up before bot finished greeting.
Details
customer-ended-call
V1
|
||||||||||
| Day 1 | 36 | 019cfd5a-08f... | 3:51 PM | 102s | $0.26 | 3/5 | KB+Transfer | Needs additional proof of address for 1583 | Caitlin | 🟡 |
|
First User Message
"I got it notarized. It's asking for additional proof of address."
First Bot Response
"For proof of address, you can provide..."
Notes
Solid KB answer. Caller became abusive after requesting transfer.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🟡 Abusive language from caller
|
||||||||||
| Day 1 | 37 | 019cfd5a-051... | 3:51 PM | 37s | $0.10 | 3/5 | Immediate Transfer | Check 1583 approval status | Jeff | 🔴 |
|
First User Message
"Can you tell me if my form fifteen eighty three has been approved?"
First Bot Response
"I'm unable to check account specific statuses..."
Notes
Bot did NOT use check_1583_status tool.
Details
customer-ended-call
V1
transfer_to_agent
🔴 Tool not invoked when it should have been
|
||||||||||
| Day 1 | 38 | 019cfd4c-f94... | 3:36 PM | 23s | $0.06 | 4/5 | Immediate Transfer | Immediate agent transfer | Jeff | |
|
First User Message
"Let me transfer me to an agent, please."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean immediate transfer.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 1 | 019d028b-f21... | 4:03 PM | 100s | $0.30 | 3/5 | KB+Transfer | UK mailbox requirements | Jeff | 🟡 |
|
First User Message
"I wanna rent my mailbox in the UK. Any special requirements?"
First Bot Response
"I don't have specific info about renting in the UK."
Notes
Correct KB gap admission. Excellent follow-up for international 1583.
Details
customer-ended-call
V1
search_knowledge_base
search_knowledge_base
transfer_to_agent
🟡 Wrong filler phrase
|
||||||||||
| Day 2 | 2 | 019d0290-695... | 4:08 PM | 23s | $0.05 | 3/5 | Immediate Transfer | Speak to representative | s | 🟡 |
|
First User Message
"Representative."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean transfer. Greeting said 'Hi, s.' — bad CRM data.
Details
customer-ended-call
V1
transfer_to_agent
🟡 Bad CRM name 's'
|
||||||||||
| Day 2 | 3 | 019d0290-f43... | 4:09 PM | 64s | $0.16 | 4/5 | KB+Transfer | How long is mailbox kept after subscription expires | Chester | 🟡 |
|
First User Message
"My subscription expired. How long does it keep the mailbox number?"
First Bot Response
"You have a ninety day grace period."
Notes
Excellent KB answer with specific details.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🟡 Wrong filler phrase
|
||||||||||
| Day 2 | 4 | 019d0293-261... | 4:11 PM | 47s | $0.14 | 2/5 | KB+Transfer | What is the legal business name | Joey | 🔴 |
|
First User Message
"What is the legal business name?"
First Bot Response
"Not specified in the search results."
Notes
Spoke before KB returned. Exposed internal logic.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🔴 Spoke before KB returned, exposed internals
|
||||||||||
| Day 2 | 5 | 019d0293-a0d... | 4:12 PM | 68s | $0.20 | 3/5 | KB+Transfer | Make a payment | Ann | 🟡 |
|
First User Message
"Trying to make a payment."
First Bot Response
"To make a payment, follow these steps..."
Notes
Good step-by-step answer but verbose.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🟡 Wrong filler, verbose
|
||||||||||
| Day 2 | 6 | 019d0294-805... | 4:13 PM | 23s | $0.06 | 4/5 | Immediate Transfer | Speak with an agent | Pamela | |
|
First User Message
"Need to speak with an agent."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean immediate transfer.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 7 | 019d0295-6ee... | 4:14 PM | 22s | $0.06 | 4/5 | Immediate Transfer | Speak to assistant | Joey | |
|
First User Message
"Speak to assistant."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean. Same caller (Joey) as Call 4.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 8 | 019d0297-5a5... | 4:16 PM | 21s | $0.05 | 3/5 | Immediate Transfer | Representative | — | 🟡 |
|
First User Message
"Representative."
First Bot Response
"Let me connect you with a support agent now."
Notes
Odd greeting: 'Hi. Yes. I'm Angie.'
Details
customer-ended-call
V1
transfer_to_agent
🟡 Odd greeting
|
||||||||||
| Day 2 | 9 | 019d0297-b12... | 4:16 PM | 49s | $0.12 | 4/5 | KB+Transfer | Change business name on account | — | 🟡 |
|
First User Message
"Can I change business name?"
First Bot Response
"To change your business name, contact customer support."
Notes
Good KB answer with verification requirement.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🟡 Wrong filler phrase
|
||||||||||
| Day 2 | 10 | 019d0297-d2e... | 4:16 PM | 21s | $0.05 | 4/5 | Immediate Transfer | Representative | Ann | |
|
First User Message
"A representative, please."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean transfer.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 11 | 019d029a-531... | 4:19 PM | 23s | $0.06 | 3/5 | Immediate Transfer | Transfer to live representative | Antoinette | 🟡 |
|
First User Message
"Can you transfer me to a live representative?"
First Bot Response
"Let me connect you with a support agent now."
Notes
Bot said 'Sorry' after caller said 'Thank you'.
Details
customer-ended-call
V1
transfer_to_agent
🟡 Awkward 'Sorry' response
|
||||||||||
| Day 2 | 12 | 019d029b-103... | 4:20 PM | 76s | $0.18 | 3/5 | Deflected | Can packages be delivered to digital address | Nikia | 🟡 |
|
First User Message
"Am I able to get packages delivered to this address?"
First Bot Response
"Yes, you can have packages delivered to your iPostal1 address."
Notes
Fully deflected. Verbose. Misread caller's 'No'.
Details
customer-ended-call
V1
search_knowledge_base
🟡 Misread 'No', verbose
|
||||||||||
| Day 2 | 13 | 019d029d-df2... | 4:23 PM | 37s | $0.08 | 2/5 | DTMF | Transfer via DTMF 0 | Jonathan | 🔴 |
|
First User Message
"DTMF: 0"
First Bot Response
"I can help with questions about your mailbox..."
Notes
DTMF 0 not recognized. Caller pressed 0 ten more times.
Details
customer-ended-call
V1
transfer_to_agent
🔴 DTMF 0 not recognized
|
||||||||||
| Day 2 | 14 | 019d02a0-c09... | 4:26 PM | 20s | $0.05 | 4/5 | Immediate Transfer | Customer service | Jean | |
|
First User Message
"Customer service."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean transfer. Shortest call.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 15 | 019d02a5-f76... | 4:32 PM | 24s | $0.06 | 4/5 | Immediate Transfer | Speak with someone at Stockbridge location | Maryland | |
|
First User Message
"Want to speak with someone at the Stockbridge location."
First Bot Response
"Let me connect you with a support agent now."
Notes
Correctly recognized as agent request.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 16 | 019d02a6-f35... | 4:33 PM | 52s | $0.13 | 4/5 | Deflected | Letter in mailbox not showing up | Michael | |
|
First User Message
"How can I track a letter not showing up?"
First Bot Response
"Create a support ticket with your name, mailbox number, and mail item."
Notes
Fully deflected. Good actionable answer.
Details
customer-ended-call
V1
search_knowledge_base
|
||||||||||
| Day 2 | 17 | 019d02a7-4fa... | 4:33 PM | 57s | $0.12 | 5/5 | Direct Answer | Can they use their own notary for 1583 | Michelle | |
|
First User Message
"Can we bring our own notary for verification?"
First Bot Response
"Local notaries are not accepted. You must use proof.com or sign in person."
Notes
Perfect answer from prompt rules. Best call of the day.
Details
customer-ended-call
V1
|
||||||||||
| Day 2 | 18 | 019d02ac-4a0... | 4:38 PM | 21s | $0.05 | 4/5 | Immediate Transfer | Customer service | Ann | |
|
First User Message
"Customer service."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean immediate transfer.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 19 | 019d02ac-c48... | 4:39 PM | 50s | $0.13 | 3/5 | KB+Transfer | Picked-up mail still showing in inbox | James | 🟡 |
|
First User Message
"Mail I picked up is still listing."
First Bot Response
"Contact your mail center directly."
Notes
Told caller to 'contact mail center by phone' — circular advice.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🟡 Circular advice
|
||||||||||
| Day 2 | 20 | 019d02ad-156... | 4:39 PM | 21s | $0.05 | 4/5 | Immediate Transfer | Hostile to AI, wants human | Jasmine | |
|
First User Message
"I hate AI."
First Bot Response
"I understand. Let me connect you with a live agent right away."
Notes
Excellent handling of hostile caller.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 21 | 019d02ae-204... | 4:40 PM | 22s | $0.05 | 4/5 | Immediate Transfer | Speak to agent (garbled audio) | Shauna | |
|
First User Message
"Can I talk to her up?"
First Bot Response
"Let me connect you with a support agent now."
Notes
Garbled transcription. Bot correctly interpreted intent.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 22 | 019d02ae-994... | 4:41 PM | 47s | $0.13 | 3/5 | KB+Transfer | Cancel account | Charles | 🟡 |
|
First User Message
"I wanna cancel my account."
First Bot Response
"To cancel, log in and go to mailbox settings..."
Notes
Standard cancellation flow.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🟡 Wrong filler phrase
|
||||||||||
| Day 2 | 23 | 019d02b3-165... | 4:46 PM | 44s | $0.13 | 4/5 | KB+Transfer | Mailbox was accidentally canceled | Christine | |
|
First User Message
"Mailbox canceled by mistake."
First Bot Response
"Contact customer support to see if it can be reinstated."
Notes
Smart adaptation — recognized actual intent.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
|
||||||||||
| Day 2 | 24 | 019d02b3-418... | 4:46 PM | 28s | $0.06 | 3/5 | DTMF | Transfer via DTMF 000000 | Jonathan | |
|
First User Message
"DTMF: 000000"
First Bot Response
"Let me connect you with a support agent now."
Notes
Same caller as Call 13. Multiple 0s recognized this time.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 25 | 019d02b3-678... | 4:46 PM | 80s | $0.17 | 3/5 | KB+Transfer | Cancel account | Melanie | 🟡 |
|
First User Message
"How do I cancel?"
First Bot Response
"To cancel, follow these steps..."
Notes
Good KB but very long. Caller asked for representative.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🟡 Verbose, wrong filler
|
||||||||||
| Day 2 | 26 | 019d02b4-ed1... | 4:48 PM | 70s | $0.16 | 4/5 | Deflected | Why mail shows up day after pickup | — | 🟡 |
|
First User Message
"Why do I get notifications the day after pickup?"
First Bot Response
"Many mail center locations upload mail once per day."
Notes
Fully deflected! Caller said 'That makes sense.'
Details
customer-ended-call
V1
search_knowledge_base
🟡 Wrong filler phrase
|
||||||||||
| Day 2 | 27 | 019d02b5-2c2... | 4:48 PM | 64s | $0.17 | 2/5 | KB+Transfer | Cancel subscription | Divina | 🔴 |
|
First User Message
"I wanna cancel my subscription."
First Bot Response
"To cancel, go to mailbox settings..."
Notes
Premature sign-off. Caller had to say 'Operator' twice.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🔴 Premature sign-off
|
||||||||||
| Day 2 | 28 | 019d02b7-898... | 4:51 PM | 20s | $0.06 | 4/5 | Immediate Transfer | Connect to live agent | Rachel | |
|
First User Message
"Connect me to the live agent."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean immediate transfer.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 29 | 019d02bc-c12... | 4:56 PM | 45s | $0.12 | 3/5 | KB+Transfer | Cancel account and get refund | Darryl | 🟡 |
|
First User Message
"Cancel my account and get a refund."
First Bot Response
"To cancel, go to settings. For refund, I'll connect you with an agent."
Notes
Good — gave steps and identified refund needs a human.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🟡 Wrong filler phrase
|
||||||||||
| Day 2 | 30 | 019d02be-5dc... | 4:58 PM | 46s | $0.12 | 2/5 | KB+Transfer | How to view submitted 1583 form | Alexandra | 🔴 |
|
First User Message
"How do I view the form I submitted?"
First Bot Response
"Hold on a sec... Sorry, a few more seconds..."
Notes
Double filler phrases.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🔴 Double filler phrases
|
||||||||||
| Day 2 | 31 | 019d02bf-760... | 4:59 PM | 21s | $0.05 | 4/5 | Immediate Transfer | Representative | Larissa | |
|
First User Message
"Representative."
First Bot Response
"Let me connect you with a support agent now."
Notes
Clean immediate transfer.
Details
customer-ended-call
V1
transfer_to_agent
|
||||||||||
| Day 2 | 32 | 019d02c0-bca... | 5:01 PM | 85s | $0.21 | 3/5 | KB+Transfer | Business name and postal address on account | Kataya | 🟡 |
|
First User Message
"Do I need to put my postal address in my business name?"
First Bot Response
"For your 1583 form, include your business name..."
Notes
'Great' triggered capabilities speech. Good KB answer.
Details
customer-ended-call
V1
search_knowledge_base
transfer_to_agent
🟡 Capabilities speech triggered
|
||||||||||
Flagged Calls 11 critical
Day 1 #21
019cfd65-4
🔴 Silence timeout — caller abandoned
Silence-timed-out. Caller confused by greeting.
Day 1 #22
019cfd64-9
🔴 Possible failed transfer — silence timeout
DTMF 0 pressed. Transfer may have failed — silence timeout.
Day 1 #37
019cfd5a-0
🔴 Tool not invoked when it should have been
Bot did NOT use check_1583_status tool.
Day 2 #4
019d0293-2
🔴 Spoke before KB returned, exposed internals
Spoke before KB returned. Exposed internal logic.
Day 2 #13
019d029d-d
🔴 DTMF 0 not recognized
DTMF 0 not recognized. Caller pressed 0 ten more times.
NOTABLE CONCERNS (22)
Day 1 #5
019cfd88-4
🟡 KB gap: Cancellation
Day 1 #6
019cfd86-e
🟡 Premature sign-off
Day 1 #9
019cfd83-4
🟡 KB gap: Cancellation
Day 1 #13
019cfd77-b
🟡 Vague answer
Day 1 #15
019cfd73-5
🟡 KB gap: Lease agreements
Day 1 #17
019cfd6b-f
🟡 KB gap: Lease agreements
Day 1 #31
019cfd5e-2
🟡 KB search not attempted
Day 1 #36
019cfd5a-0
🟡 Abusive language from caller
Day 2 #1
019d028b-f
🟡 Wrong filler phrase
Day 2 #2
019d0290-6
🟡 Bad CRM name 's'
Day 2 #3
019d0290-f
🟡 Wrong filler phrase
Day 2 #5
019d0293-a
🟡 Wrong filler, verbose
Day 2 #8
019d0297-5
🟡 Odd greeting
Day 2 #9
019d0297-b
🟡 Wrong filler phrase
Day 2 #11
019d029a-5
🟡 Awkward 'Sorry' response
Day 2 #12
019d029b-1
🟡 Misread 'No', verbose
Day 2 #19
019d02ac-c
🟡 Circular advice
Day 2 #22
019d02ae-9
🟡 Wrong filler phrase
Day 2 #25
019d02b3-6
🟡 Verbose, wrong filler
Day 2 #26
019d02b4-e
🟡 Wrong filler phrase
Day 2 #29
019d02bc-c
🟡 Wrong filler phrase
Day 2 #32
019d02c0-b
🟡 Capabilities speech triggered
Key Ratios
Deflection Rate
5.7%
Calls fully resolved by bot
Transfer Rate
51.4%
Calls escalated to human agent
KB Search Rate
55.7%
Calls where KB was queried
Personalization Rate
68.6%
Calls with name greeting
Green Calls
49.3%
Rating 4-5 (of rated calls)
Yellow Calls
39.1%
Rating 3 (of rated calls)
Red Calls
11.6%
Rating 1-2 (of rated calls)
Knowledge Base Gaps 7 topics
Lease Agreements
open
No KB articles about mailbox lease agreements or how to obtain copies
Refs: Day 1 #17, Day 1 #15
Account Cancellation
open
Cancellation process vague — bot gives generic steps, callers want agent to cancel
Refs: Day 1 #9, Day 1 #5, Day 2 #22, Day 2 #25, Day 2 #27, Day 2 #29
Billing Disputes
open
No KB articles for billing disputes or double-billing
Refs: Day 1 #12
International Requirements (UK)
partial
Bot has US international info but not UK-specific rental details
Refs: Day 2 #1
Legal Business Name
open
Bot couldn't find iPostal1's legal business name
Refs: Day 2 #4
View Submitted 1583 Form
open
No KB article on how to view a previously submitted 1583 form
Refs: Day 2 #30
App Troubleshooting
partial
Basic troubleshooting exists but no deep technical guidance
Refs: Day 1 #22, Day 1 #20