Web and Mobile Development: Anatomy of an HTTP Request
Bill Mongan
What Happens When You Make a Web Request?
- When you navigate to www.google.com from your browser, what happens exactly?
Making an HTTP Request: The Client
- Obtain the IP Address associated with www.google.com
- Make a TCP connection to that server IP address
- Communicate using the HTTP protocol to request
/index.html
Making an HTTP Request: The Server
- Determine which file (URL) the client has requested
- Open that file from the hard drive
- Send the reply back over the TCP connection via the HTTP protocol
Details
- How do we determine an IP address from a host?
- How do we communicate which page we want to access in a methodical way?
- What if the page we obtain has images or other media in it?
- What if we have to send state information like a product number?
- How does the server maintain state like a shopping cart?
- What if we have to send sensitive information like a password?
HTTP Request
GET /index.html HTTP/1.1
Option: Value
<blank line>
<body input data>
<blank line>
- Why do we need a blank line?
HTTP Response
HTTP/1.1 200 OK
Option: Value
<blank line>
<web page data>
<blank line>
- Have you ever seen an HTTP error code?
HTTP is Stateless
- This makes HTTP versatile (why?)
- Why is it easier to catch a bus than a flight?
HTTP is Stateless
- If you send two requests, each looks like a brand new request
- This makes HTTP versatile (why?)
HTTP: Try It
Wireshark
- Tracing with Wireshark
HTTPS
- Telnet completed the DNS Request, but we were redirected by the server to use https. Let’s try again:
HTTPS
HTTPS
HTTPS
HTTPS: Second Request
- We were redirected again to PHP
- A human would be frustrated, but computers don’t mind! However, these requests do take time.
HTTPS
Passing Data
- What if your data includes non-textual binary data, like an image?
- Remember that everything is represented by binary bits of 1’s and 0’s.
- We can convert raw binary data into text!
Base64 Encoding
- Base64 Encoding represents binary sequences as characters
var data = "This is a test";
var encoded = btoa(data);
var decoded = atob(encoded); // === data
Re-Imagining a Web Request
- What if we didn’t read from the hard drive to gather the response data, but ran a program instead?
Re-Imagining a Web Request
- What if we considered HTTP inputs as function parameters, and the HTTP response body as the function result?
- The response actually comes from memory, not the disk, and is dynamically generated!
Re-Imagining a Web Request
- If that request was instead for a resource, and the server invoked an operation on a resource (for example, a SQL query), and returned its result, we get an instance of a RESTful Web Service.
- You could even run the program from Telnet.
- Your browser wouldn’t know the difference!
Re-Imagining a Web Request
Re-Imagining a Web Request
- Just don’t change those URL’s, if you can avoid it!
- Why not?
HTTP Clients
- In the past (and present), we developed mobile applications that invoke these web services for you
- More recently, web-based applications run in your web browser and can be run across platforms (including desktop applications!)
HTTP Clients
const https = require('https');
https.get('https://api.weatherusa.net/v1/obs?station_id=KLOM', (resp) => {
let data = '';
resp.on('data', (chunk) => {
data += chunk;
});
resp.on('end', () => {
const respobj = JSON.parse(data);
console.log(data);
console.log("Weather at " + respobj[0].station_id);
console.log("Sky: " + respobj[0].sky);
console.log("Temperature: " + respobj[0].t_f);
var sunset = new Date(0);
sunset.setUTCSeconds(respobj[0].sunset_t);
console.log("Sunset time: " + sunset);
});
}).on('error', (e) => {
console.log(e.message);
});
- Save as
httpclient.jsand run with:node httpclient.js
HTTP Clients
[{"station_id":"KLOM","stname":"Wings Field","net":"METAR","lat":"40.1399993896"
,"lon":"-75.2699966431","elev":"92","cond":"","sky":"CLR","obstime":"37 minutes
ago (2020\/08\/31 11:35:00 EDT)","obstime_t":"1598888100","t_f":71,"td_f":61,"p_
hpa":1019,"p_in":"30.08","precip":null,"snow_depth":null,"wspeed_mph":6,"wgust_m
ph":null,"wdir_compass":"ENE","hum":"72%","chill":74,"feelslike":74,"sunrise_t":
1598869664,"sunset_t":1598916852,"night":false,"wx_icon":"clear","wx_str":"Clear
"}]
HTTP Clients
Weather at KLOM
Sky: CLR
Temperature: 71
Sunset time: Mon Aug 31 2020 19:34:12 GMT-0400 (Eastern Daylight Time)
- How would you print the current humidity?
HTTP Clients with cURL
Progressive Web Apps
API Endpoints
- See if you can find some REST API’s for your favorite applications
Web and Mobile Development: Anatomy of an HTTP Request
Bill Mongan
