Web and Mobile Development: Anatomy of an HTTP Request

Bill Mongan

What Happens When You Make a Web Request?

  • When you navigate to www.google.com from your browser, what happens exactly?

Making an HTTP Request: The Client

  • Obtain the IP Address associated with www.google.com
  • Make a TCP connection to that server IP address
  • Communicate using the HTTP protocol to request /index.html

Making an HTTP Request: The Server

  • Determine which file (URL) the client has requested
  • Open that file from the hard drive
  • Send the reply back over the TCP connection via the HTTP protocol

Details

  • How do we determine an IP address from a host?
  • How do we communicate which page we want to access in a methodical way?
  • What if the page we obtain has images or other media in it?
  • What if we have to send state information like a product number?
  • How does the server maintain state like a shopping cart?
  • What if we have to send sensitive information like a password?

HTTP Request

GET /index.html HTTP/1.1
Option: Value
<blank line>
<body input data>
<blank line>
  • Why do we need a blank line?

HTTP Response

HTTP/1.1 200 OK
Option: Value
<blank line>
<web page data>
<blank line>
  • Have you ever seen an HTTP error code?

HTTP is Stateless

  • This makes HTTP versatile (why?)
    • Why is it easier to catch a bus than a flight?

HTTP is Stateless

  • If you send two requests, each looks like a brand new request
  • This makes HTTP versatile (why?)

HTTP: Try It

HTTP Request with Telnet

Wireshark

Wireshark Trace

HTTPS

  • Telnet completed the DNS Request, but we were redirected by the server to use https. Let’s try again:

HTTPS

HTTPS Request with Telnet

HTTPS

HTTPS Request with Telnet

HTTPS

HTTPS Request with Telnet

HTTPS: Second Request

  • We were redirected again to PHP
    • A human would be frustrated, but computers don’t mind! However, these requests do take time.

HTTPS

HTTP Request with Telnet

Passing Data

  • What if your data includes non-textual binary data, like an image?
  • Remember that everything is represented by binary bits of 1’s and 0’s.
  • We can convert raw binary data into text!

Base64 Encoding

var data = "This is a test";
var encoded = btoa(data);
var decoded = atob(encoded); // === data

Re-Imagining a Web Request

  • What if we didn’t read from the hard drive to gather the response data, but ran a program instead?

Re-Imagining a Web Request

  • What if we considered HTTP inputs as function parameters, and the HTTP response body as the function result?
    • The response actually comes from memory, not the disk, and is dynamically generated!

Re-Imagining a Web Request

  • If that request was instead for a resource, and the server invoked an operation on a resource (for example, a SQL query), and returned its result, we get an instance of a RESTful Web Service.
  • You could even run the program from Telnet.
    • Your browser wouldn’t know the difference!

Re-Imagining a Web Request

Re-Imagining a Web Request

  • Just don’t change those URL’s, if you can avoid it!
    • Why not?

HTTP Clients

  • In the past (and present), we developed mobile applications that invoke these web services for you
  • More recently, web-based applications run in your web browser and can be run across platforms (including desktop applications!)

HTTP Clients

const https = require('https');

https.get('https://api.weatherusa.net/v1/obs?station_id=KLOM', (resp) => {
  let data = '';

  resp.on('data', (chunk) => {
    data += chunk;
  });

  resp.on('end', () => {
    const respobj = JSON.parse(data);

    console.log(data);
    console.log("Weather at " + respobj[0].station_id);
    console.log("Sky: " + respobj[0].sky);
    console.log("Temperature: " + respobj[0].t_f);
    
    var sunset = new Date(0);
    sunset.setUTCSeconds(respobj[0].sunset_t);
    console.log("Sunset time: " + sunset);
  });
}).on('error', (e) => {
  console.log(e.message);
});
  • Save as httpclient.js and run with: node httpclient.js

HTTP Clients

[{"station_id":"KLOM","stname":"Wings Field","net":"METAR","lat":"40.1399993896"
,"lon":"-75.2699966431","elev":"92","cond":"","sky":"CLR","obstime":"37 minutes
ago (2020\/08\/31 11:35:00 EDT)","obstime_t":"1598888100","t_f":71,"td_f":61,"p_
hpa":1019,"p_in":"30.08","precip":null,"snow_depth":null,"wspeed_mph":6,"wgust_m
ph":null,"wdir_compass":"ENE","hum":"72%","chill":74,"feelslike":74,"sunrise_t":
1598869664,"sunset_t":1598916852,"night":false,"wx_icon":"clear","wx_str":"Clear
"}]

HTTP Clients

Weather at KLOM
Sky: CLR
Temperature: 71
Sunset time: Mon Aug 31 2020 19:34:12 GMT-0400 (Eastern Daylight Time)
  • How would you print the current humidity?

HTTP Clients with cURL

cURL Web Service Client Call

Progressive Web Apps

API Endpoints