Sunday, May 28, 2023
Home   >   Cover Story   >   The Dangers of an Evertec Monopoly

The Dangers of an Evertec Monopoly

By on January 28, 2016

Analyzing Puerto Rico’s ATM Network

In the late afternoon of Jan. 9, during the tail-end of the holiday shopping season and the beginning of back-to-school sales for the new semester, most of Puerto Rico’s debit system stopped working. From around 5 p.m. to 7 p.m., people were unable to use their debit cards to take cash out of automated-teller machines (ATMs), while thousands of stores and restaurants couldn’t process debit transactions on their point-of-sale (POS) terminals.

As a result, commercial activity on the island, at least for a couple hours, came to a near-complete halt; this was an unwelcome development considering Puerto Rico’s already weakened economic and fiscal state. It was also an unprecedented failure for the island’s main debit network—called ATH, short for “A Toda Hora” (At Every Time)—and a rare setback for the network’s owner and operator, Evertec.

A San Juan-based company that began as a subsidiary of Popular Inc. in 1988, Evertec has risen to become a leading player among transaction-processing service providers in the Caribbean and Latin America, serving 19 countries in the region.

In September 2010, Apollo Management LLC, a private equity investor, acquired a 51% interest in Evertec. The following year, the company expanded its services to the Mexican market, and in 2013, Evertec became the first technology company in Puerto Rico to be listed on the New York Stock Exchange (NYSE), under the ticker symbol EVTC. By that time, it had already expanded its footprint in various markets in Central America, as well as Aruba, Curaçao and Colombia, to name a few.

“The company manages a system of electronic payment networks that process more than 2.1 billion transactions annually, and offers a comprehensive suite of services for core bank processing, cash processing and technology outsourcing,” reads a blurb on the company’s website.

However, in the space of just two hours, the firm’s once-sterling reputation has been tarnished. Following the network blackout on Jan. 9, Evertec has come under fire, with public officials and private sector leaders not only questioning the firm’s security measures and network capacity, but also its business practices, partly due to the ubiquitous nature of its ATH debit network in Puerto Rico. Monopoly allegations have even been bandied about the company amid an ongoing P.R. Senate investigation.

In a development that proved perhaps more startling than the network outage itself, Evertec kept relatively mum about the incident for several days afterwards, which fueled further speculation. “It’s very strange that they would choose to remain silent this way,” said a person close to Evertec who spoke on condition of anonymity. “On previous occasions, minor malfunctions were usually explained away and nothing more came of it. This time it’s different, both in the scale of what happened and the way they have handled it.”Evertec Network Operating Center

Once it re-established the ATH system and for days after, the company only made the following statement through social media: “In Evertec, we work hard to maintain a quality service and we apologize for any inconvenience that this situation may have caused.” No information was given about the cause of the outage.

About a week later, the company published a similarly short message on its Facebook page: “In Evertec, the security of our information is our priority. That is why we comply with PCI [Payment Card Industry] standards to protect our clients’ data, providing the security they deserve.” Calls by various news outlets, including Caribbean Business, for the company to provide additional details went unanswered until recently.

Commerce Interrupted

About two weeks after the fact, Evertec representatives sat down with Caribbean Business and other media outlets, during which Carlos Ramírez, Evertec’s executive vice president of business solutions, estimated that about 200,000 electronic transactions were affected during the two-hour service interruption.

Regarding the potential effects the interruption may have caused in the Puerto Rico economy, local economists interviewed by Caribbean Business concurred that the incident’s economic impact was marginal at best.

“It’s hard to estimate the economic impact because many of the missed sales transactions using debit cards during those two hours could have been completed with cash or checks,” said Heidie Calero, president of H. Calero Consulting Group.

José Joaquín Villamil, chairman of Estudio Técnicos Inc., agreed with Calero, adding that some of those missed transactions during the two-hour mishap were merely delayed and could have been completed once the Evertec systems were back online. Villamil stressed instead  the lessons learned from Evertec’s service interruption. “Technology makes us more vulnerable. The more interconnected we are, the more turbulent the interruption and the more vulnerable we become,” he said.

Before computers and POS systems became the norm, merchants used regular cash-register machines. If something happened to the cash register, the mishap was isolated only to that merchant  since there were no interconnections between other merchants or suppliers, Villamil said. With the high amount of interconnectivity these days due to the use of technology, if a supplier suffers a service interruption, many merchants and their clients are affected, he noted.

José Alameda, an economics professor at the University of Puerto Rico’s Mayagüez campus, gave a different perspective on the incident by providing a rough calculation of the possible monetary value of the losses stemming from the service interruption. He did this by combining Evertec’s data of 200,000 uncompleted transactions during the two-hour service interruption with transaction estimates of the sales & use tax from the Treasury Department.

With 77,770 POS on the island, according to Treasury, and with each POS averaging $3,162.35 in sales per day (amounting to $316.23 in average sales per hour) the estimated value of the losses for those 200,000 uncompleted transactions over two hours came up to $126.4 million, according to Alameda’s calculations.

An Anomaly

Alan Cohen, Evertec’s executive vice president of marketing & communications, apologized for the outage on behalf of the company. “A two-hour service interruption doesn’t comply at all with the level of service we are used to providing to our clients,” he said.

As to the underlying cause, “first of all, it wasn’t an attack or a hack,” Evertec’s Ramírez told Caribbean Business. “No data was compromised and there was no theft involved.”

What took place, the exec said, was “an anomalous failure” in one of the company’s main servers that in turn caused a massive outage in the ATH network on two fronts. “The first front was on the POS side,” Ramírez explained. Specifically, there is a point in which a transaction from a store or restaurant switches over to the system of the bank that holds the debit card’s account. “It was at that switch point that the failure took place,” he said.

At the same time, an outage occurred in the network’s ATM terminals, an area that falls more directly under Ramírez’s purview. Evertec employees stationed at the company’s Network Operating Center first noticed the breakdown at 4:27 p.m., when alarms began to go off in the system. “There were too many cards being declined; that rapidly told us something was wrong,” the exec said. When pressed about the exact nature of the malfunction, Ramírez said it was due to “software failure.”

In a system as complex as Evertec’s, conflicts between different components and/or glitches are almost inevitable. The key, Ramírez noted, lies in the level of redundancy that the system has; in other words, the system’s capacity to recover quickly. In most instances, the system’s back-up systems are enough to recover in no time, with no visible effects. However, on Jan. 9, the glitch affected a lot of components at the same time, and it was of such a magnitude that the system was unable to self-correct.

At first, Ramírez and his crew checked to see if it was the result of a hack, most probably a common type of attack called a DDoS (Distributed Denial of Service). After eliminating that possibility, as well as other attacks such as a Trojan or a virus, the team deliberated whether to essentially move the whole system to secondary facilities that the company has at an undisclosed location in the U.S. mainland.

“We evaluated the situation and decided that we wouldn’t suffer further delays in carrying out the system’s recovery and  wouldn’t have to rely on the mainland facilities,” Ramírez said, adding that switching over to the secondary platform would have taken roughly the same time as the company ultimately took in re-establishing the network.

Afterward, it was basically a matter of rebooting the server, not unlike what one does at home with a malfunctioning personal computer. By 6:28 p.m., according to the company, the network was mostly up, particularly the POS side. “There was a general perception that the outage took longer to fix because many stores and restaurants assumed the system was down and were telling people that debit cards weren’t working well into the night, even though we were reporting on the latest developments through our various platforms,” Ramírez noted.

Some observers speculated that high-volume activity stemming from back-to-school sales played a factor in overloading the ATH network, especially since the government had implemented a brief sales tax-free period at the time. However, Ramírez dismissed the notion. “We were experiencing a typical Saturday volume sales-wise when the malfunction took place,” he said.Ruben Piñero

When asked about what steps are being taken to prevent a similar incident from taking place, Ramírez said that after carrying out an in-depth investigation, the company has begun implementing recommendations given by the firm’s internal work team and hired outside experts. “The key is to reduce recovery time significantly,” the exec noted. “Just to make sure, we are operating at double the capacity that we currently need.”

“In 10 years, we have had a network reliability of 99.9%,” Evertec’s Cohen added. “However, no system is completely flawless; it just so happens that on Jan. 9, that 0.01% came into play.”

Probe on the Way

Despite their best efforts to clear any doubts, the company’s relative silence on the matter during the first few weeks following the incident has prompted further scrutiny. For instance, Popular Democratic Party (PDP) Sen. Luis Daniel Rivera Filomeno, who chairs the Senate Consumer Affairs Committee, filed Senate Resolution 1321 mere days after the service interruption.

The resolution calls for an investigation into the outage itself, the rates that Evertec charges clients and whether the company represents a monopoly. The Senate approved the resolution late last week, with the first hearings scheduled for this past Tuesday, after this story went to press.

Rivera Filomeno told Caribbean Business that many businessowners have complained for years that Evertec charges too much for their service. “They have pointed out that the fees are way above those charged in the U.S. mainland,” he noted. “It has reached the point in which many establishments choose to go cash-only in their transactions. This not only limits consumers, but also opens the door for tax evasion and even money laundering. It’s a domino effect.”

Rubén Piñero, president of the United Retailers Association, said it was necessary to launch an investigation into Evertec’s alleged monopolistic practices. “For us retailers, having one company with almost complete control of the island’s commercial and banking transactions is a dangerous scenario.”

To this, Rivera Filomeno added: “Evertec controls such a huge part of the market that it’s almost impossible for store owners to go to anyone else for their electronic transaction needs, even though there are six more similar companies registered here in Puerto Rico.” Two of those companies, Pay-tech and Multi-business, were scheduled to testify at the Senate hearings this week.

“Eventually, we will have the participation of all seven companies in the segment, as well as officials from the Consumer Affairs Department and the Monopolistic Affairs Office at the Justice Department,” Rivera Filomeno noted. “The intent is to have a clear picture, and if we deem that it is necessary to draft a bill that would take care of the problem, then that is what we will do.”Evertec out

Meanwhile, House Resolution 1108, which was filed by PDP Rep. Javier Aponte Dalmau, was also approved this past week. Of note is that this resolution was filed in October 2014, and originally sought to look into Evertec’s alleged monopolistic practices. Aponte Dalmau chairs the House’s Small Business, Commerce, Industry & Telecommunications Committee.

While Evertec’s execs said they wouldn’t respond to monopoly allegations until they testify in the hearings, they insisted they have been operating in an open, competitive environment. “First of all, we aren’t alone in the segment; there are also the likes of Visa and MasterCard in the field,” Evertec’s Cohen explained. “They not only deal with credit cards, but also with debit, the only difference being that ATH is pin-based and their cards are signature-based.”

By the same token, Ramírez disputed the idea that all ATMs on the island are the property of ATH and hence, Evertec’s. “As it turns out, we don’t operate most of the ATMs on the island,” he said. “There are about 12 companies who supply these terminals, which are mostly available at small stores, and we don’t operate those.”

And while ATH is by far the most well-known debit network on the island, several measures prevent Evertec or any other company from being the sole gatekeepers of the island’s electronic transactions, the execs went on to say. “By federal law, each debit card must have access to at least two networks,” Ramírez said. “A typical one may have the ATH logo and a Visa or MasterCard logo. Each logo represents a network.

“Federal law also regulates the rates that we may be able to charge, so that particular argument also falls flat,” he added.

Finally, as to the question about why Evertec took so long to provide details to the public about the service interruption, the key may lie in the firm’s status as a public company, with the additional pressures that this implies. “The company needed to be absolutely sure of what happened and why it happened before we went public with it,” Cohen said. “We needed to be very careful.”

You must be logged in to post a comment Login