Getting to Know Fiddler: Part I

If you've done any amount of web development on Windows, you've probably used a program called "Fiddler" a few times. Fiddler is a small proxy server that allows you to see all of the requests made by your system. Sadly, for most people, I'd guess that that is all that Fiddler is. It's a shame because Fiddler offers up so much more power than that.

So, let's fix this. For the next several weeks, we're going to explore some of the under noticed functionality that Fiddler provides us. We're going to examine the functionality that past me would have loved to have known about.

Remote Debugging

One of the first bits of power is realizing that Fiddler isn't a proxy server for your desktop, but rather a proxy server that just happens to run on your desktop. Assuming we provide the configuration, Fiddler will be more than happy to proxy any traffic it receives, such as from another computer, mobile device, or even video game console. We can enable remote proxying by clicking "Tools", "Fiddler Options...", and then the "Connections" tab. On this page, we need to enable the checkbox for "Allow remote computers to connect."

Enable remote connections

With this stage of our configuration out of the way, we can restart Fiddler and move to another device on the network. The device we want to examine will need a bit of manual configuration to configure proxying. For specific device configuration instructions, you'll need to head to Google. 1

In the case of my iPhone, I opened Settings, clicked on my network, and then scrolled to the bottom of the page where I was able to enter the IP address of my desktop, and the port we configured for our proxy, in this case, 8888.

Proxy configuration and... Pinterest?!

Now time for some web traffic to prove that our proxy works. In this case, I launched the Pinterest app. 2 Immediately Fiddler lit up with requests and responses. We can use this technique to allow debugging of devices where no debugging proxy is natively available. When we're done, we can simply go back in to our device and remove the proxy configuration. 3

Look at our beautiful web service!

This technique can be invaluable for diagnosing a computer where installing Fiddler isn't an option, as well as developing for any form of mobile device.

Now Fiddler is no longer limited to showing us the things we browse from our computers. Still, our current outlook doesn't consider it much more than a glorified Firebug. Next week we'll start on the functionality that truly sets it apart.

Footnotes

1 You should be able to search "How to setup a proxy for" your device. The only device that I regularly use that I couldn't find a page for was the original Kindle.
2 Yep. Pinterest. It is actually okay for a guy to be on Pinterest. If you feel the obligation to become indignant though, focus not on the fact that I chose the Pinterest app, but that the Pinterest app traffic was HTTP. Not necessarily bad, just interesting.
3 If you forget to remove the proxy configuration from your device, you'll likely see that you are unable to browse the web after closing Fiddler, or leaving the network that Fiddler is running on. You also won't be able to pull up this warning, so there's that.

Tags: 

Stubbing Web Services with Sinatra

Introduction

The concept of dependencies seems rather straight forward. If Thing A depends on Thing B, then we can understand that we can't possibly use Thing A until we have Thing B. It's not even a development principle really, it's more of an "even small kids realize this" kind of principle. It's sort of a universal given, unless you happen to be Gallifreyan.

Taking that a step further, if you want me to make an application for you using some web services you've published, it might be prudent to give me access to those services, right? "Yeah, we haven't actually written the services yet, but we've figured out the API". Oh. Awkward. We can mitigate some of the problem with unit tests, but how do we test actual hands on usage? There are a few possibilities here, but one that I'm rather partial to is mocking up some services. Even provided access, there are some benefits to making a mock of an external service, such as easier testing of fault tolerance, and avoiding unnecessary calls to paid services.

Getting Started

While we could throw together some services in almost any language with any number of frameworks, as a matter of convenience, I'm going to go with Sinatra on Ruby. I'm assuming that Ruby is already installed on the local system.

To start, we need to create a Gemfile to list our dependencies. For this project we'll end up using two gems.

  1. source "https://rubygems.org"
  2. gem "sinatra"
  3. gem "json"

Out of the three lines in our Gemfile, the lines fall into two varieties. The first is the source line. This will tell bundler where to install the gems from. After that we have the gem lines. Each of these specify a gem, and optionally a version to install. With our populated Gemfile, we can run bundle install to download and install our new gems.

  1. # bundle install
  2. Fetching gem metadata from https://rubygems.org/...........
  3. Fetching additional metadata from https://rubygems.org/..
  4. Resolving dependencies...
  5. Using json 1.8.1
  6. Installing rack 1.5.2
  7. Installing rack-protection 1.5.3
  8. Installing tilt 1.4.1
  9. Installing sinatra 1.4.5
  10. Using bundler 1.6.2
  11. Your bundle is complete!
  12. Use `bundle show [gemname]` to see where a bundled gem is installed.

Great. With Sinatra installed, we can continue to create our first simple web services.

Hello World!

We're going to need one more file in order to get our services setup. Create and name this one app.rb. With it created, we can have our first set of services running with only five lines. 1

  1. require 'sinatra'
  2.  
  3. get '/' do
  4. "Hello World"
  5. end

Now run ruby app.rb. At this point you should be able to hit http://localhost:4567 and see "Hello World".

Look at our beautiful web service!

Okay, it works, but how? Line 1 is self-explanatory enough: we're loading the Sinatra gem. But what about after that? Line three is where the true magic happens. Sinatra has defined a number of new methods for us, including post, patch, delete, option, and among others, get. On line three we're invoking the method get with two parameters: a string '/', and a Proc2. Effectively, we're associating a path with an anonymous function. When we navigate to "/", Sinatra runs our Proc and displays the output to us.

So far, so good. Let's increase the scope of our hello world a bit.

A Slightly Bigger Hello World

  1. require 'sinatra'
  2.  
  3. get '/users' do
  4. "Hello World"
  5. end
  6.  
  7. post '/users' do
  8. [403, params[:name]]
  9. end

Our new Hello World isn't much bigger, but it is now big enough to show off several concepts for us. For starters, we've now seen post, our first verb other than get. As you probably suspect, this associates a post to /users with the given Proc. Line seven has a second detail for us as well: /users is defined twice, with different verbs. Routes evaluated in the order that they were defined. First match wins.

There are two more things worth noting in our example. First, is the return value on line eight. The new return value is an array that follows the format [http_status, message_body]. Since our other examples returned a 200 status code, we were able to ignore explicitly returning it. The last item of interest is the params array. When a request posts data, the params array will contain all of the values sent to the server.

In the case of our example, posting to /users with a form item named 'name' will result in a 403 status, and a message body that matches the value we posted.

A Stubbed Service

Now we've got JUST enough Sinatra knowledge under our belt to create a dummy payment API that we can use. Our api will expect a post to /payment that contains a number and an expiration date. Invalid requests get a 400. If our card is valid, we'll get a 200 and a JSON success, otherwise, we'll get a 403 and a JSON failure message.

  1. require 'sinatra'
  2. require 'json'
  3.  
  4. post '/payment' do
  5. return [400, "Missing Card Number"] unless params[:number]
  6. return [400, "Missing Expiration"] unless params[:expiration]
  7.  
  8. valid = is_valid?(params[:number], params[:expiration])
  9. status = valid ? 200 : 403
  10.  
  11. [status, { "success" => valid }.to_json]
  12. end
  13.  
  14. def is_valid?(number, expiration)
  15. number == "0000-0000-0000-0000" and expiration == "10/15"
  16. end

And to test out our fake services, we'll make a few calls to it.

  1. curl http://localhost:4567/payment -d "number=0000-0000-0000-0000;expiration=10/16" -w " %{http_code}"
  2. {"success":false} 403

  1. curl http://localhost:4567/payment -d "number=0000-0000-0000-0000;expiration=10/15" -w " %{http_code}"
  2. {"success":true} 200

  1. curl http://localhost:4567/payment -d "number=0000-0000-0000-0000" -w " %{http_code}"
  2. Missing Expiration 400

Excellent.

Conclusion

Our final example is still a bit small, but I believe that it gives a proper idea of the kinds of items that we can easily stub out. Ideally we'd have guaranteed access to all the services we need at all stages of development. That doesn't always happen though. Still, with the knowledge of using stubbed services in our toolkit, lack of availability might not be the blocker that it once was. 3

Footnotes

1 Technically we could condensed our sample down to two lines, if we were to sacrifice all semblance of readability. Let's not.
2 For my non-ruby friends, a Proc is conceptually similar to an anonymous method. Similar.
3 Granted, if one of the problems is that you can't access the APIs you need, I suspect there will be other problems that Ruby might not be able to help...

I <3 DevOps because I <3 Code

If you've spent any time looking at the items that I post, you've probably gathered that I'm a developer, but that I have a rather strange habit of not talking about code. I talk about build automation, source control, compiler output, virtualization, proxy servers, and continuous integration, but I barely talk about matters of code. So if I love coding, why do I spend my time talking talking about DevOps rather than code? A necessity of my love of code is a love of DevOps.

It's rare to meet a developer that has no interest in code. Coding is like an irresistable calling, a siren song: hard to ignore even in the face of logic. As such, the desire of "I'd rather be coding" has sunk many a developer into a pit of small repeating problems. These small problems become an inescapable quicksand that stymies further progress to the point of being nothing more than violent thrashing.

So where does DevOps fit in to this? To me, DevOps is the philosphy that it's better to stop coding and fix an underlying problem for an hour than it is to waste the full day in the name of "code". More colloquially, "An ounce of prevention is worth a pound of cure." 1 And this is why I focus so much energy on DevOps: I work with smart people doing cool stuff and we'd all "rather be coding". We don't have time to waste on manual builds, miscommunication, and manual configuration.

DevOps addresses the issues that keep my team and I from doing the thing that we love. DevOps allows us to develop for the long term, augmenting our ability to develop; It empowers us spend more time on the actual problems that matter.

So yes, I love DevOps, because I wouldn't have the time to code without it.

Footnotes

1 For my metric friends: "28.3 grams of prevention is worth 453 grams of cure." :P

Resetting the Root Password on Linux

The drunken scribblings of a mad man. Possibly some kind of ancient cuneiform. A random page out of the Middle Earth phone directory. Whatever it was, it was the server password, not that it matters now: the little Post-It note reminder had long since thrown itself into the waste paper bin. Somewhere a landfill sat, knowing our password and chuckling away at our misfortune.

It seems that our hero is in a bit of trouble, if only slightly self-inflicted. (Repeat after me: "Passwords do not go on sticky notes.") All is not lost, however. Assuming we have physical access to the server, this is more of an inconvenience than a brick wall. Assuming we haven't encrypted our user directory, we should be able to reset the password and get going again with little effort. Provided that we have a LiveCD, that is.

Start by booting the live media and mounting the target drive that our existing Linux is installed on. One of the easiest ways to do this is with the disk for Ubuntu Server.

With our root mounted from the LiveCD, we should be able to reset the password with only one command. From an existing terminal session, run the following, where "/target" is the path that your root filesystem is mounted to. Voila!

  1. chroot /target passwd

How it works

In Linux when we want to change a user's password we run passwd. passwd by itself presents us with a chicken-and-egg problem though: we can't use passwd to change the password for root until we login as root.

What if we just ran passwd from the LiveCD then? That wouldn't really help us either. That would simply change the password for the root user of the live session. The problem with just running passwd is that the live environment isn't the environment that needs updating.

The magic here is chroot $directory $command. In this case, the util chroot makes /target appear to be the root while it runs passwd. Thus, passwd is run against the root file system of the correct environment.

Tags: 

Pages

3120 days since since I met you..
 
 
Powered by Drupal