We here at TopMark Funding covered WEX fuel cards as one of the best fuel cards in the market, but it appears there is a downside to being the most popular fuel card in the country. On the night of February 4th, 2020, WEX cards were not accepted at diesel pumps across the country.
Hitting the Limit
We live in a society where everything is uploaded to the cloud. Fuel purchases and card memberships are no exception. WEX and its parent company EFS stored their business information on the IBM Informix, which IBM calls “a scalable, integrated database with self-managing capabilities, optimized for OLTP and Internet of Things (IoT) data.”
Clearly, the database system was not self-managing enough.
WEX sent out the following message to its customers the next afternoon:
“Last night we experienced an issue with our IBM Informix database where a ‘chunk’ of our tables and data exceeded the maximum Terabyte allowable by IBM. This was an unknown limit by EFS/Fleet One and IBM and required significant troubleshooting by IBM to identify the problem.”
By being the most commonly used fleet card in the country, WEX’s fuel card data exceeded capacities that IBM did not even know it had. WEX went on further to say that they had four systems of redundancy and hitting the data limit wall happened with all four of them at roughly the same time. WEX says they are taking additional steps to prevent this catastrophe from happening again.
The Damage is Done
The bulk majority of cases of truck drivers being unable to purchase fuel from their WEX cards happened sometime between 9 PM on February 4th to 3 PM on February 5th (EST). During these eighteen hours, drivers were forced to either use a different method of payment or wait until the issue subsided.
Adding fuel to the fire of this situation was the customer support that people claim to have received from WEX. On EFS’ Facebook page, customers complained of being unable to purchase fuel with their cards and being put on hold for extended periods of time waiting for a customer service representative.
Because the FMCSA only allows truckers to be on the clock for 14 hours a day, it is safe to assume that if a driver needed to stop for fuel with only a WEX card in your pocket, his or her day was effectively over (and FMCSA spokesperson said this system malfunction does not constitute an emergency, and as such, no extra Hours of Service time was given).
Considering most commercial drivers are paid based upon mileage, this was especially detrimental to people who pulled in to refuel right after the catastrophe had started.
The WEX card incident shows that even the most expansive of databases have hard data limits, even if the company that owns the servers to hold the data do not know what those limits are.
If you are an owner-operator or a fleet manager, you can use this event as a convincing argument to keep at least two methods of payment on hand. If you work for another as a company driver, maybe ask your employer for emergency cash, or keep some of your own hidden in the cabin and ask for reimbursement with a receipt should the scenario ever arise again. It is certainly better than losing an entire day to some network malfunction beyond your control.
Another more drastic solution is to make the switch from diesel to another fuel source such as electricity. Electric vehicles have some issues in regards to charging time, but the technology for faster fuel charging times is catching up. You could also consider using a semi-truck that runs on a form of natural gas.
But in the end, perhaps the simplest solution is also the easiest: a can of emergency diesel fuel to keep your truck going some extra miles while the cloud managers fix their servers.
Article courtesy of TopMark Funding.