Dear Concord Customer,

We sincerely apologize for any interruption in service you may have experienced on Monday, June 8th, and want to provide you with the following update and timeline information.  

At approximately 11:00 PDT, a degradation of service was noted with Concord’s inbound fax service, prompted by an anomalous spike in activity and subsequent difficulty with error handling at the carrier level.

Mitigation efforts by Concord Platform Engineers successfully resolved the issue causing inbound service degradation at approximately 16:00 PDT, and services have remained fully restored for 48+ hours. 

In addition to its successful mitigation efforts, Concord engineers are continuing work with our telephony partners to implement additional long-term measures to safeguard against recurrence.

As all systems have remained fully restored for 48+ hours, please report any further issues with your to premiumsupport@concord.net, so that we may further assist you.

Concord Platform Operations isolated the primary issue preventing us from processing inbound faxes on impacted numbers.  Adjustments were made during the course of troubleshooting, which appear to have alleviated the issue at approximately 4:00pm PST.  

Platform Engineering will remain in conference with our carrier to research the root cause of the problem and ensure it is understood, but inbound fax traffic is being processed successfully at this time.

Support will continue to actively monitor our inbound platform to ensure it remains stable, and will immediately escalate directly to the VP of Operations if we detect any anomalous behavior.

Thank you for your patience and understanding as we worked through this issue.  Support will provide further updates once we have isolated the root cause and confirmed that it is resolved in its entirety.

Concord Network Operations are still working to diagnose the root cause of the inbound fax failures currently impacting our clients.  Incident SAMI-23275 has been opened to track this issue.

Faxes to the impacted range of numbers are reaching our primary ingress carrier network, but fail in the negotiation process which results in a fast busy signal or dead air.  Enterprise Support and Platform Operations have confirmed that all network appliances on the Concord platform are online and fully functional, so we have opened ticket 18921597 with our carrier to ask they investigate.  This ticket was escalated at the earliest opportunity, and Concord Engineers are working directly with technical support from the carrier's escalation group at this time.

Unfortunately, we do not have an ETA for resolution at this time, though all necessary resources and personnel remain engaged and dedicated to this issue.  Support will update via email and the Network Health dashboard with any further information as soon as it becomes available.

Finally, please understand that Concord is fully aware of the critical nature of the fax traffic that we are privileged to process for our clients, and we sincerely apologize for any impact caused by this outage.

Concord is investigating service degradation on its Inbound Platform.  We will update this incident as information becomes available.