Getting to Know Fiddler: Part II

Last week we started looking at the web proxy Fiddler, or the "Fiddler Web Debugger" as it unfortunately bills itself. While the name is accurate, I can't help but feel that it is a bit of an understatement, much like calling the Hulk "somewhat tempermental"1, or MacGyver "good at assembling things". Sure, Fiddler is a debugger, but it's less of a simple tool and more the web equivalent of a swiss army knife that contains an entire Home Depot2.

Before we get into some of the more advanced features, we'll need to figure out how to generate requests of our own.

Manually Crafting Requests

Two weeks ago we wrote a simple web service with Sinatra. Our web service had one endpoint at /payment that used POST and had two parameters: expiration and number. At the time, we tested our web service with cURL. We'll test that same service here, this time using Fiddler.

To create a new request from scratch, we'll need to click the "composer" tab.

The composer tab consists of five elements. While this may not seem like much, it provides practically everything that we might care about for a request. We'll start in the first field by choosing the verb we want to use for our request, POST in this case. In the second field, we'll put the URL of our service. This brings us to the third field which lets us choose the exact version of the HTTP spec to use. While this could be useful in theory, I have yet to ever need this field.

The fourth field is reserved for any request headers we want to add, one per line. Here we could request a content type, specify a user agent, or pretend like anyone actually honors "Do Not Track". Part of the beauty is what we don't have to specify: the host and content-length. Fiddler automatically fills these in for us, rather than making us calculate the length by hand.

The last field is used exclusively for verbs that allow form data, such as POST and PUT. Unlike the headers above that were one per line, the contents of this field are form encoded.

With our request assembled, click Execute to send off the request. The left column will now have our request and response in it, ready for us to examine by double-clicking.

Reusing Requests

While it can be useful to manually create a request, the true power that Fiddler provides through the composer tab is with the ability to base our new request of off a captured request. With the composer tab opened, find the captured request that you would like to use as a starting point. Click it in the list and drag the request into the composer. Doing so will cause the composer to turn green until the request is dropped in. Now all of the fields will have been filled in for us. If we wanted, we could reissue the original request simply by hitting Execute. Alternatively we could tweak any of the fields if we simply wanted to use the capture as a starting point.

Personally, I think that beats cURL any day.

Conclusion

Now we know how to remotely debug using Fiddler, and how to craft new requests from scratch or from captured ones. Those techniques are quite useful, and are some that I use almost everyday. This is just the beginning though. Still, that should be enough for one week.

Footnotes

1 You won't like me when I'm generally perturbed. Hulk jostle!
2 The pictured knife is $1,349, but hey, there's free shipping. That makes it a deal. Right?

Tags: 

Getting to Know Fiddler: Part I

If you've done any amount of web development on Windows, you've probably used a program called "Fiddler" a few times. Fiddler is a small proxy server that allows you to see all of the requests made by your system. Sadly, for most people, I'd guess that that is all that Fiddler is. It's a shame because Fiddler offers up so much more power than that.

So, let's fix this. For the next several weeks, we're going to explore some of the under noticed functionality that Fiddler provides us. We're going to examine the functionality that past me would have loved to have known about.

Remote Debugging

One of the first bits of power is realizing that Fiddler isn't a proxy server for your desktop, but rather a proxy server that just happens to run on your desktop. Assuming we provide the configuration, Fiddler will be more than happy to proxy any traffic it receives, such as from another computer, mobile device, or even video game console. We can enable remote proxying by clicking "Tools", "Fiddler Options...", and then the "Connections" tab. On this page, we need to enable the checkbox for "Allow remote computers to connect."

Enable remote connections

With this stage of our configuration out of the way, we can restart Fiddler and move to another device on the network. The device we want to examine will need a bit of manual configuration to configure proxying. For specific device configuration instructions, you'll need to head to Google. 1

In the case of my iPhone, I opened Settings, clicked on my network, and then scrolled to the bottom of the page where I was able to enter the IP address of my desktop, and the port we configured for our proxy, in this case, 8888.

Proxy configuration and... Pinterest?!

Now time for some web traffic to prove that our proxy works. In this case, I launched the Pinterest app. 2 Immediately Fiddler lit up with requests and responses. We can use this technique to allow debugging of devices where no debugging proxy is natively available. When we're done, we can simply go back in to our device and remove the proxy configuration. 3

Look at our beautiful web service!

This technique can be invaluable for diagnosing a computer where installing Fiddler isn't an option, as well as developing for any form of mobile device.

Now Fiddler is no longer limited to showing us the things we browse from our computers. Still, our current outlook doesn't consider it much more than a glorified Firebug. Next week we'll start on the functionality that truly sets it apart.

Footnotes

1 You should be able to search "How to setup a proxy for" your device. The only device that I regularly use that I couldn't find a page for was the original Kindle.
2 Yep. Pinterest. It is actually okay for a guy to be on Pinterest. If you feel the obligation to become indignant though, focus not on the fact that I chose the Pinterest app, but that the Pinterest app traffic was HTTP. Not necessarily bad, just interesting.
3 If you forget to remove the proxy configuration from your device, you'll likely see that you are unable to browse the web after closing Fiddler, or leaving the network that Fiddler is running on. You also won't be able to pull up this warning, so there's that.

Tags: 

Stubbing Web Services with Sinatra

Introduction

The concept of dependencies seems rather straight forward. If Thing A depends on Thing B, then we can understand that we can't possibly use Thing A until we have Thing B. It's not even a development principle really, it's more of an "even small kids realize this" kind of principle. It's sort of a universal given, unless you happen to be Gallifreyan.

Taking that a step further, if you want me to make an application for you using some web services you've published, it might be prudent to give me access to those services, right? "Yeah, we haven't actually written the services yet, but we've figured out the API". Oh. Awkward. We can mitigate some of the problem with unit tests, but how do we test actual hands on usage? There are a few possibilities here, but one that I'm rather partial to is mocking up some services. Even provided access, there are some benefits to making a mock of an external service, such as easier testing of fault tolerance, and avoiding unnecessary calls to paid services.

Getting Started

While we could throw together some services in almost any language with any number of frameworks, as a matter of convenience, I'm going to go with Sinatra on Ruby. I'm assuming that Ruby is already installed on the local system.

To start, we need to create a Gemfile to list our dependencies. For this project we'll end up using two gems.

  1. source "https://rubygems.org"
  2. gem "sinatra"
  3. gem "json"

Out of the three lines in our Gemfile, the lines fall into two varieties. The first is the source line. This will tell bundler where to install the gems from. After that we have the gem lines. Each of these specify a gem, and optionally a version to install. With our populated Gemfile, we can run bundle install to download and install our new gems.

  1. # bundle install
  2. Fetching gem metadata from https://rubygems.org/...........
  3. Fetching additional metadata from https://rubygems.org/..
  4. Resolving dependencies...
  5. Using json 1.8.1
  6. Installing rack 1.5.2
  7. Installing rack-protection 1.5.3
  8. Installing tilt 1.4.1
  9. Installing sinatra 1.4.5
  10. Using bundler 1.6.2
  11. Your bundle is complete!
  12. Use `bundle show [gemname]` to see where a bundled gem is installed.

Great. With Sinatra installed, we can continue to create our first simple web services.

Hello World!

We're going to need one more file in order to get our services setup. Create and name this one app.rb. With it created, we can have our first set of services running with only five lines. 1

  1. require 'sinatra'
  2.  
  3. get '/' do
  4. "Hello World"
  5. end

Now run ruby app.rb. At this point you should be able to hit http://localhost:4567 and see "Hello World".

Look at our beautiful web service!

Okay, it works, but how? Line 1 is self-explanatory enough: we're loading the Sinatra gem. But what about after that? Line three is where the true magic happens. Sinatra has defined a number of new methods for us, including post, patch, delete, option, and among others, get. On line three we're invoking the method get with two parameters: a string '/', and a Proc2. Effectively, we're associating a path with an anonymous function. When we navigate to "/", Sinatra runs our Proc and displays the output to us.

So far, so good. Let's increase the scope of our hello world a bit.

A Slightly Bigger Hello World

  1. require 'sinatra'
  2.  
  3. get '/users' do
  4. "Hello World"
  5. end
  6.  
  7. post '/users' do
  8. [403, params[:name]]
  9. end

Our new Hello World isn't much bigger, but it is now big enough to show off several concepts for us. For starters, we've now seen post, our first verb other than get. As you probably suspect, this associates a post to /users with the given Proc. Line seven has a second detail for us as well: /users is defined twice, with different verbs. Routes evaluated in the order that they were defined. First match wins.

There are two more things worth noting in our example. First, is the return value on line eight. The new return value is an array that follows the format [http_status, message_body]. Since our other examples returned a 200 status code, we were able to ignore explicitly returning it. The last item of interest is the params array. When a request posts data, the params array will contain all of the values sent to the server.

In the case of our example, posting to /users with a form item named 'name' will result in a 403 status, and a message body that matches the value we posted.

A Stubbed Service

Now we've got JUST enough Sinatra knowledge under our belt to create a dummy payment API that we can use. Our api will expect a post to /payment that contains a number and an expiration date. Invalid requests get a 400. If our card is valid, we'll get a 200 and a JSON success, otherwise, we'll get a 403 and a JSON failure message.

  1. require 'sinatra'
  2. require 'json'
  3.  
  4. post '/payment' do
  5. return [400, "Missing Card Number"] unless params[:number]
  6. return [400, "Missing Expiration"] unless params[:expiration]
  7.  
  8. valid = is_valid?(params[:number], params[:expiration])
  9. status = valid ? 200 : 403
  10.  
  11. [status, { "success" => valid }.to_json]
  12. end
  13.  
  14. def is_valid?(number, expiration)
  15. number == "0000-0000-0000-0000" and expiration == "10/15"
  16. end

And to test out our fake services, we'll make a few calls to it.

  1. curl http://localhost:4567/payment -d "number=0000-0000-0000-0000;expiration=10/16" -w " %{http_code}"
  2. {"success":false} 403

  1. curl http://localhost:4567/payment -d "number=0000-0000-0000-0000;expiration=10/15" -w " %{http_code}"
  2. {"success":true} 200

  1. curl http://localhost:4567/payment -d "number=0000-0000-0000-0000" -w " %{http_code}"
  2. Missing Expiration 400

Excellent.

Conclusion

Our final example is still a bit small, but I believe that it gives a proper idea of the kinds of items that we can easily stub out. Ideally we'd have guaranteed access to all the services we need at all stages of development. That doesn't always happen though. Still, with the knowledge of using stubbed services in our toolkit, lack of availability might not be the blocker that it once was. 3

Footnotes

1 Technically we could condensed our sample down to two lines, if we were to sacrifice all semblance of readability. Let's not.
2 For my non-ruby friends, a Proc is conceptually similar to an anonymous method. Similar.
3 Granted, if one of the problems is that you can't access the APIs you need, I suspect there will be other problems that Ruby might not be able to help...

I <3 DevOps because I <3 Code

If you've spent any time looking at the items that I post, you've probably gathered that I'm a developer, but that I have a rather strange habit of not talking about code. I talk about build automation, source control, compiler output, virtualization, proxy servers, and continuous integration, but I barely talk about matters of code. So if I love coding, why do I spend my time talking talking about DevOps rather than code? A necessity of my love of code is a love of DevOps.

It's rare to meet a developer that has no interest in code. Coding is like an irresistable calling, a siren song: hard to ignore even in the face of logic. As such, the desire of "I'd rather be coding" has sunk many a developer into a pit of small repeating problems. These small problems become an inescapable quicksand that stymies further progress to the point of being nothing more than violent thrashing.

So where does DevOps fit in to this? To me, DevOps is the philosphy that it's better to stop coding and fix an underlying problem for an hour than it is to waste the full day in the name of "code". More colloquially, "An ounce of prevention is worth a pound of cure." 1 And this is why I focus so much energy on DevOps: I work with smart people doing cool stuff and we'd all "rather be coding". We don't have time to waste on manual builds, miscommunication, and manual configuration.

DevOps addresses the issues that keep my team and I from doing the thing that we love. DevOps allows us to develop for the long term, augmenting our ability to develop; It empowers us spend more time on the actual problems that matter.

So yes, I love DevOps, because I wouldn't have the time to code without it.

Footnotes

1 For my metric friends: "28.3 grams of prevention is worth 453 grams of cure." :P

Pages

3129 days since since I met you..
 
 
Subscribe to Syndicate
Powered by Drupal