Uncovering the Hidden Roadblock in Payment Processing
We were met with non-answers - assuring us that the problem must lie on our end. Every developer knows how discouraging poor customer support can be, combined with a lack of relevant resources on the web. Disappointed, but not surprised, we reached out to fellow developers at Stack Overflow in search of answers. Turns out we were not the only ones with this issue and in fact, most of the other developers struggling with it were using cloud-based services as well.
With the new-found knowledge, we decided to share the post with customer support along with the IDs of failing requests and they could no longer blame us for a faulty implementation or incorrect cyphers. We finally received a positive response, announcing a solution, alongside a description of the root cause of the problem.
All of the people struggling with the issue were using some sort of a web-based function that would not have a static IP assigned to it. The third-party system, that the payment provider was using, had a protection that would prevent known bad actors from performing certain operations, based on the IP address. We can only assume that some of the IPs in Amazon’s pool were used to perform malicious actions and that is why we were getting blocked, whenever a function was run on one of those machines. The solution was to turn off that firewall for the clients, but as a precaution, we decided to create a dedicated set of IPs for our functions, so that a similar problem would not happen in the future.