Cybersource is a subsidiary of Visa, and is one of the largest providers of online credit card payment services. As any developer who has worked with Cybersource’s Silent Order POST service can tell you, it’s not the easiest service to work with. It provides a great deal of flexibility, but that comes at the cost of you having to write a good deal of your own code in order to use it. Setting up automated testing is also extremely difficult.

Last year I completed a Cybersource project for the University of Pennsylvania, and that project provided the inspiration for 2 Ruby gems, to simplify working with Cybersource: Cybersourcery, and Cybersourcery Testing. There’s also a demo project, so you can see an example of how to use them in a Rails project.

The readme files provide detailed documentation of their features and how to use them. So rather than repeat that information here, let’s take a look at why these gems are necessary in the first place. There’s a lot to cover, so I’ll discuss the testing gem in this post, and Cybersourcery in the next one.

Writing tests that can be repeated and automated provides benefits such as improving the design of your code (if you’re doing test-driven development) and catching regressions early (when changes to your code inadvertently introduce bugs). This can be challenging with 3rd party services, as we don’t want to call those services every time we run our test suite. VCR is a gem that helps with this problem: by recording requests and responses, it allows you to play back those responses in your tests, rather than making real-life calls in every test run.

Unfortunately, Cybersource makes this kind of testing especially difficult. There are 3 different servers involved in processing a transaction through Cybersource, and the key difficulty is that one of them is at a fixed URL that is not easy to swap out in the test environment. Cybersource calls this URL the “Customer Response Page.” It is saved on the Cybersource server as part of the merchant profile, so it cannot be updated dynamically. If you are a developer attempting to test Cybersource transactions, this diagram illustrates the scenario:

                     +                    +                   +
                     |     Developer's    |    Cybersource    |  "Customer Response"
     User's browser  |     test server    |    test server    |         server
  +------------------+--------------------+-------------------+---------------------+

    Request credit
      card form
          +
          |
          +----------->   Respond with
                        credit card form
                               +
                               |
     Submit form <-------------+
          +
          |
          +------------------------------>   Process and log
                                              transaction;
                                            generate "Customer
                                              Response" form
                                                    +
                                                    |
    Hidden "Customer <------------------------------+
    Response" form is
      automatically
     submitted by JS
          +
          |
          +--------------------------------------------------->   Process submission;
                                                                 generate response page
                                                                         +
                                                                         |
        Display  <-------------------------------------------------------+
     response page

So, what the heck is going on here? The first few steps makes sense, but then when you submit the credit card payment form to Cybersource, things start to seem strange. What happens is that Cybersource sends a seemingly blank page to your browser. But it only appears for a second, as it contains a hidden form, which is immediately and automatically submitted to the “Customer Response Page.” This is the page where users are sent when transactions are complete. You provide the URL for this page when setting up your merchant profile in the Cybersource Business Center. This is a page you create and host yourself – you can use it to show users a “thank you” message, log information about the transaction, etc.

So why doesn’t Cybersource simply redirect to your response page after processing the transaction? Why this peculiar reliance on a hidden form? The reason is that conventional redirects use the GET method, which is meant for idempotent requests. An idempotent request is one that can be safely repeated, which definitely does not apply to a credit card transaction, or logging it. So Cybersource’s forms appropriately use the POST method, which is meant for non-idempotent requests. This is why, if you submit a POST form, and then click “back” in your browser, and try to submit the form again, your browser will warn you, and ask if you really want to submit the form again.

In the case of Cybersource, this is a thorny problem. Trying to do a POST redirect has issues, for these reasons. A redirect isn’t really appropriate anyway: the Cybersource server does some work when it receives the user’s credit card submission (charging the user’s card), and then your response page may also do some work when it receives the hidden form submission (such as logging details of the transaction). These are distinct activities, so – while having two forms may seem odd – it’s a viable solution. Cybersource came up with this before asynchronous requests were a common practice (which is a big part of the reason it’s harder to work with than newer services like Stripe).

The Cybersourcery Testing gem makes it possible to set up automated, repeatable tests in this complex environment. It provides a “translating proxy” server, running on Sinatra, which has middleware to manage the requests and responses between the 3 servers involved in Cybersource transactions. Middleware is especially useful in this situation, as it allows us to modify requests and responses before they reach the application’s code.

In order to explain how the gem works, let’s first take a look at its dependencies:

• The Rack Proxy gem is a Rack app which allows you to rewrite requests and responses. It’s very open-ended and is designed to be subclassed, so you can implement rewrite rules appropriate to your project.
• The Rack::Translating Proxy gem inherits from the Rack Proxy gem, and provides an implementation that’s suitable for the Cybersource use case. We need to provide it with a target_host URL, which indicates where the proxy server should redirect requests. We also need to provide it with a request_mapping, which indicates what strings to find in the requests and responses, and what to change them to. It uses a hash format, so that on requests, the keys are translated to the values, and on responses, the values are translated to the keys.

The Cybersourcery Testing gem inherits from the Rack::Translating Proxy gem, and implements the methods described above. Specifically:

For the target_host, we provide the URL of the Cybersource testing site. So if the proxy server is running at http://localhost:5556, and the target_host is https://testsecureacceptance.cybersource.com, requests to http://localhost:5556/some/path will be redirected to https://testsecureacceptance.cybersource.com/some/path. The gem also hooks into VCR, allowing us to record transactions that pass through the proxy server, for use in subsequent automated tests.

This is a simplified version of the request_mapping implementation, using hard-coded values for clarity:

def request_mapping
  {
    # local test server                Cybersource's "Customer Response Page" URL
    'http://localhost:1234/confirm' => 'http://your-site.com/confirm'
  }
end

A Cybersource transaction in this environment looks like this:

1. The credit card form is submitted and the proxy server receives it.
2. Based on the target_host, the proxy server passes the request through to the actual Cybersource test server. If the transaction was previously recorded with VCR, VCR will instead play back the recording of the transaction.
3. Cybersource (or VCR) sends a hidden form back to the browser, which is automatically submitted via JavaScript to the “Customer Response Page” URL. The middleware’s request_mapping will rewrite the URL of the form’s action, causing the form to instead submit to the local test server.

The upshot is, the gem handles all this complexity so you don’t have to. By following the setup steps in the readme, you can get a robust test environment setup for Cybersource without breaking a sweat. The Cybersourcery Testing gem offers other features as well, such as reporting of undocumented Cybersource errors. Check out the README to learn more!