Spoofing User-Agent with Chrome’s WebRequest API

Whenever you ask for a page on the web, your request has a lot of data attached to it. One part of your request (the “user-agent” string) describes your software environment—usually your operating system and your browser’s name and version number. Like many things on the web, this data is a convention, not a law. You are free to modify it however you want. This practice is known as user-agent spoofing.

If you’re ready to give it a try yourself (and you have Google Chrome), you can find the extension (and installation instructions) here. If you’re a developer interested in modifying headers from a Chrome extension, read on!

Implementation Details

Update II (1/24/12) It looks like Chrome 17 will ship with support for modifying the User-Agent string. (Click here for the Chromium issue and here for more info on the feature.) This renders the dedicated user-agent switching extension obsolete, but the approach outlined here may still be relevant in the context of a larger extension. Also remember the WebRequest API has many applications beyond this simple use-case!

Update I (1/3/12) In Chrome 17, the WebRequest API will lose its “experimental” designation and change slightly. This article, originally published in August 2011, has been updated to reflect these changes. For the more diff-minded developer, I’ve updated the example project with a single commit which documents the specific changes.

We’ll be using Chrome’s WebRequest API to accomplish this task. First off, you will have to request permission to use it in your extension’s manifest.json file:

”permissions”: [ “webRequest” ]

…although in our specific case, we’ll need some additional permissions: one for “blocking” all requests until we’ve completely processed them, and another for our extension to operate on all URLs. That means our manifest file will actually look more like this:

"permissions": [ "webRequest", “webRequestBlocking”, “<all_urls>” ]

I’ll be focusing on the “onBeforeSendHeaders” event for now. Behaviors like request blocking and redirecting can be achieved with other event handlers, but those will have to wait for another post.

You can bind your own listener to this event in your extension’s background page. Here is an example binding which outlines the necessary steps:

background.js

  // The 'reqestFilter' parameter allows you to only listen for
  // certain requests. Chrome 17 requires that, at the very least,
  // it defines the URLs you wish to subscribe to. In the general
  // case, we want to subscribe to all URL's, so we'll explicitly
  // declare this requirement.
var requestFilter = {
    urls: [ "<all_urls>" ]
  },
  // The 'extraInfoSpec' parameter modifies how Chrome calls your
  // listener function. 'requestHeaders' ensures that the 'details'
  // object has a key called 'requestHeaders' containing the headers,
  // and 'blocking' ensures that the object your function returns is
  // used to overwrite the headers
  extraInfoSpec = ['requestHeaders','blocking'],
  // Chrome will call your listener function in response to every
  // HTTP request
  handler = function( details ) {

    var headers = details.requestHeaders,
      blockingResponse = {};

    // Each header parameter is stored in an array. Since Chrome
    // makes no guarantee about the contents/order of this array,
    // you'll have to iterate through it to find for the
    // 'User-Agent' element
    for( var i = 0, l = headers.length; i < l; ++i ) {
      if( headers[i].name == 'User-Agent' ) {
        headers[i].value = '>>> Your new user agent string here <<<';
        break;
      }
      // If you want to modify other headers, this is the place to
      // do it. Either remove the 'break;' statement and add in more
      // conditionals or use a 'switch' statement on 'headers[i].name'
    }

    blockingResponse.requestHeaders = headers;
    return blockingResponse;
  };

chrome.webRequest.onBeforeSendHeaders.addListener( handler, requestFilter, extraInfoSpec );

…that should be enough to get you up and running.

Still Some Uncertainty

If you’re anything like me, after reading the documentation and the above example, you still have some questions. I’ll do my best to answer them:

What are the contents of requestHeaders? The API documentation gives hints at what it doesn’t contain, but my experience has been varied. Here’s a printout of what I’ve seen, although I’ve received confirmation from Chrome developers that the contents are variable:

requestHeaders.json

[
        {"name":"Pragma","value":"no-cache"},
        {"name":"User-Agent","value":"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/14.0.835.15 Safari/535.1"},
        {"name":"Accept","value":"text/css,*/*;q=0.1"},
        {"name":"Cache-Control","value":"no-cache"},
        {"name":"Referer","value":"http://mikepennisi.com/"},
        {"name":"Accept-Encoding","value":"gzip,deflate,sdch"},
        {"name":"Accept-Language","value":"en-US,en;q=0.8"},
        {"name":"Accept-Charset","value":"ISO-8859-1,utf-8;q=0.7,*;q=0.3"},
        {"name":"Cookie","value":"__utma=138710319.518446708.1311977381.1311977381.1311977381.1; __utmz=138710319.1311977381.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); ci_session=6YkLJxXZRFbZ5DLlpFCL8KG3cnggPjSvEtP7VCNRTv4JLphOrHGUqPX3pfOuURl0cfbMb9Gu4%2FcNnsxZyiTNxH%2B0hXZ4pElZush0NJNHHD7ReOV2hoHWRaDX1ikw4vWamoyB8WvLftxotbKX4ibITIwqBlQ0mqq%2BcIQRn5uW0ueFgwzt43b5Q6KBGYJ2RMXZrrT%2FgB0ViV8dofvXNmQ8qFueXOlotnnWx1KcTWyyiWDOIia2%2Bm5ZuknlbmvFMk5czOMf5y70S4oaK7iup7gAlaPSVD9%2FZE%2BzslzuCvvyiXM%3D"}
]

How do I detach a listener? Although currently undocumented, chrome.webRequest.onBeforeSendHeaders.removeListener() seems to work. It may be safer (for now) to begin your listener function by checking an externally-scoped listenerIsActive flag which you can set/unset from outside the function.

background-toggle.js

var listenerIsActive = true,
  requestFilter = {
    urls: [ "<all_urls>" ]
  },
  extraInfoSpec = ['requestHeaders','blocking'],
  handler = function( details ) {

    var headers = details.requestHeaders,
      blockingResponse = {};

    if( !listenerIsActive ) {
      // If you take this approach, just make sure you return
      // the headers, or else your "disabled" listener will
      // delete all headers from future requests!
      return {requestHeaders: details.requestHeaders};
    }

    // Your code here...
  };

chrome.webRequest.onBeforeSendHeaders.addListener( handler, requestFilter, extraInfoSpec );

How can I use requestFilter to control which requests the listener responds to? The requestFilter parameter optionally lets you filter by tab ID and window ID. You can also specify a URL pattern to filter by–the pattern schema is documented here.

Hopefully, this has been a useful introduction to Chrome’s WebRequest API. As you have seen, it is now possible to spoof the user-agent string from an extension. The API seems to be stabilizing in preparation for its coming promotion from the “experimental” namespace. Be sure to reference the official documentation for the most up-to-date details.