JS 171 Study Guide

Overview

What is the internet?

The internet is a network of networks. It consists of network infrastructure (all the devices, routers, cables, hubs, switches, etc that it needs to function) AND protocols that guide the transmission of messages and allow the infrastructure to function.

What is a network?

At a minimum, a network is two or more devices connected so they can exchange info/communicate. The internet is a network of interconnected networks.

Local networks

LAN — Local area network. Multiple devices connected via a bridging device/hub/switch via cables.

WLAN — Wireless local area network. Multiple devices connected via a wireless hub.

What is a router?

While a hub or switch sends signals between connected devices in a single network, a router sends signals between network hubs — they direct the traffic between networks.

What is a protocol?

Protocols are the rules governing data transmissions. Different protocols address different aspects of network communication, or address the same aspect of network communication in different ways. Some examples are:

• HTTP: Hypertext Transfer Protocol (message structure)
• TCP: Transmission Control Protocol (message transfer between applications)
• UDP: User Datagram Protocol (message transfer between applications)
• IP: Internet Protocol (message transfer between networks)
• TLS: Transport Layer Security (cryptographic protocol)
• Ethernet (message transfer between devices on a network)

There are two important models for thinking about how these protocols work.

OSI Model: Organized by layer function

• Application
• Presentation
• Session
• Transport
• Network
• Data Link
• Physical

Internet Protocol Suite Model: Organized by communication scope

• Application (top 3 layers of OSI)
• Transport (transport layer of OSI)
• Internet (network layer of OSI)
• Link (bottom 2 layers of OSI)

PDU

What are they? Protocol Data Units. These are the blocks of data at each layer that transferred across networks. They are different at different layers of network communication/with different protocols. Some examples of PDU include HTTP messages, UDP datagrams, TCP segments, IP packets, Ethernet frames, etc.

What are the components of PDU?

• Headers — provide the metadata of a PDU (address, length, flags etc)
• Data Payload — the data being transported across the network using a protocol at a given layer. The data payload is made up of the entire PDU from the layer above it (the data payload for a TCP segment is an HTTP request, the data payload for an IP packet is a TCP segment, etc)
• Trailers/Footers — optional. provides more metadata.

Encapsulation

How is encapsulation implemented? Data from one layer gets hidden in the payload of the layer below.

What are the benefits?

• Separation between layers: the protocols at any given layer do not need to understand the implementation of protocols at other layers.
• Abstraction: Allows us to work on higher level programs without worrying about implementation at underlying layers. I can work with SMTP or HTTP or FTP on the application layer without worrying about what protocol to use on the transport or network layers.

Physical

What happens in the physical layer?

Bits are transferred physically through light, electrical, or radio signals.

What is Bandwidth?

The amount of data that can be sent in a given unit of time (usually 1 second).

What is Latency?

The total the amount of time (in ms) it takes for a signal to get from its starting point to its destination. There are several types of delay that can add to this latency, including:

• Propagation delay — time to deliver message based on speed and distance… Light signals have less propagation delay than electrical signals
• Transmission delay — time to push message into link/router
• Processing delay — time to process message at each link
• Queueing delay — time waiting to be processed at each link

What is last-mile latency? Signals taking more time near network endpoints. (e.g. getting the network signal from your ISP’s network to your home/office network)

What is Round-trip time? Sum of request and response time.

Link/Data Link

Ethernet Protocol. How does it operate?

Ethernet enables communication between devices in a network. Interface between physical and logical networks.

Ethernet Frames

The PDU of the Link layer of the network. These add logical structure to binary code, which allows ethernet-enabled devices to determine which parts of a received frame are addresses and which are data. Frames take an IP packet as their data payload.

What is an inter-frame gap? The amount of time between frames to prepare a device for the next frame. It adds to the transmission delay.

MAC Addressing

Mac addresses are assigned to devices when they are manufactured. These addresses allow frame transmission between multiple devices on a local network (connected by a switch or hub). A HUB will receive a frame and send it to all the devices in network. The recipients will check if the intended address matches their own address. If they match, the device will receive the frame, otherwise it will discard the frame. A SWITCH will receive a frame, and compare addresses to its own switch port, then pass the message on only to the intended recipient.

Problems with MAC Addressing: It’s not scaleable. Since MAC addresses are physical and non-hierarchical, switches would have to track all MAC addresses at the same time, which is impossible. MAC addressing only works on a local network.

Internet/Network

What does IP do?

Facilitates message transfer between hosts on different networks. Routing packets via IP addresses. Packets are the PDU of IP, and they take TCP segments/UDP datagrams (or individual fragments of segments and datagrams) as their data payload.

What’s an IP Address?

IP addresses are assigned when a device joins a network — they are logical addresses, rather than physical addresses like the MAC address. IPv4 addresses are hierarchical and are 32 bits in length (4 numbers between 0 + 255). The first 3 numbers represent the network address — so you can route packets to a router in a specific network segment, which can then forward the message to the another router in a specific subnet, etc. (e.g. 109.156.106.128 might be the network address that can send to subnet IP addresses 109.156.106.129 and 109.156.106.130.) We’re about to run out of IPv4 addresses, and will start using IPv6, which uses 128 bit addresses and has a different header structure for packets)

How do messages get lost at this level?

• TTL (time to live): If a packet bounces around too long, it gets dropped. The TTL tracks the number of hops.
• checksum mismatch: The sending device and receiving device use the same algorithm to generate a “checksum” value. If a received package’s checksum doesn’t match the receiving device’s generate value, the package gets dropped.

Transport

What is multiplexing?

Multiplexing is sending multiple signals over a single channel. Demultiplexing is turning a single channel back into multiple signals.

What’s a port number?

An identifier for a specific process running on a host. It’s an integer in the range 0–65535. When combined with an IP address, it forms a socket — the communication endpoint for messages. An example of a socket is 216.3.128.12:8080.

Common ports:

• Well known ports 0–1023 (used by servers)
• Registered ports 1024–49151 (assigned as requested by private entities)
• Dynamic/private ports 49152–65535 (assigned by OS and used by client machines. can’t be registered for a specific use)

What is the difference between a connectionless and a connection-oriented system?

• A connectionless system does not need to verify a connection before sending. A receiving object at one end (defined by a socket) listens for incoming messages from any source at any time in any order, and processes/responds to the incoming messages as they arrive.
• A connection-oriented system verifies that a connection is present before sending or recieving messages. TCP is one example of a connection-oriented system that verifies a connection using a three-way-handshake. Another example of a connection oriented process would be: a receiving object at one end (defined by a socket) listens for incoming messages from any source at any time in any order. When a message arrives, it creates a new socket object which only listens for messages from the sender of the recieved message, as defined by the sender’s socket (source IP + source port). This helps with communication reliability because it lets you put in place rules for message order, message acknowledgements, retransmission, and general reliability.

TCP & UDP

What is TCP?

Transmission control protocol. This is a connection-oriented system which abstracts reliable data communication. It uses a three way handshake to establish a connection, and sends data through TCP segments (which contain application layer PDU as their data payload). Segments track the source port, destination port, sequence number, acknowledgment number, SYN ACK flags (1-bit boolean fields), window size (how much data a receiver can accept at any point during connection), and checksum (see IP checksum).

Benefits of TCP

• Encapsulation
• Multiplexing
• Reliability

Downside of TCP

• Latency overhead due to the many rounds of communication as a connection is established. Leads to slower performance.
• Head-of-line blocking due to ordered messages being dropped. Any segments that were meant to arrive after the dropped message have to wait in buffer until the dropped segment is retransmitted. Leads to queueing latency.

How does TCP increase reliability of message transfer?

• In-order delivery
• Message acknowledgment
• Retransmission of lost data
• Data integrity
• De-duplication
• Flow control
• Network congestion

What is a three-way-handshake with TCP?

• Sender sends TCP segment with SYN flag set to 1
• Receiver responds with TCP segment with SYN ACK flags set to 1
• Sender responds with TCP segment with ACK flag set to 1. Then proceeds to send application data. Receiver can respond with data only after it receives the ACK segment.

Why does a three-way-handshake exist? To provide a reliable connection for message transmission. Allows for message acknowledgement, retransmission, and in-order delivery.

What is UDP?

User Datagram Protocol is a connectionless system that provides multiplexing and encapsulation, but not reliability. It does not guarantee message delivery, order, congestion avoidance/flow control, or connection state. Uses datagrams as PDU, which track source port, destination port, length, + checksum.

Benefits of UDP

• Faster performance: avoids the latency overhead and head of line blocking of TCP. Great for processes that require speed more than reliability — like voice and video calling.
• Flexible: lets you implement your own rules at the application level for sequencing or flow control.

Similarities and Differences

What do they have in common? Encapsulation + multiplexing.

What’s the difference? TCP provides reliability. UDP is faster and more flexible.

What are some things that can go wrong in the transport layer?

How does TCP deal with flow control and network congestion?

• Flow control: Receiver lets sender know how much data it is willing to receive at any given point using ‘window size’ header. Any data beyond that window size gets stored in the buffer. If the buffer is getting full, the receiver can change the window size.
• Network congestion: When transmitted data exceeds network capacity (like if a router is overwhelmed and its buffer overflows and it drops packets), TCP will notice that there are lots of retransmissions happening and reduce the transmission window size.

How does UDP deal with flow control and network congestion?

• It doesn’t!

Application

What’s the difference between internet and web?

Internet is infrastructure for inter-network communication + the protocols guiding that communication. Web is a service that lets you access an information system via the internet through URLs. Web uses HTML, URIs, and HTTP technologies to create uniformity:

• HTML: Hypertext markup language. How resources on the web are structured.
• URI: Uniform resource identifier. string of characters to identify a particular resource.
• HTTP: Hypertext transfer protocol. rules to provide uniformity to the way web resources are transferred between aplications.

DNS — How does it work?

DNS, or Domain Name System, is a distributed database which translates human-friendly domain names like www.google.com into IP addresses. DNS keeps track of of domain names and their corresponding IP addresses in a huge hierarchy of DNS servers. When you enter a URL into a browser, the browser creates an HTTP request, packages it, and sends it to your device’s network interface. The device has a cache storing important information locally. It checks its cache for the IP address of that domain. If it doesn’t find the IP address in the cache, it makes a request to DNS. DNS sends back the right IP address, and the packaged HTTP request goes over the Internet where it is directed to the server with the matching IP address (which contains the web content). The receiving server accepts the request and sends an HTTP response (plaintext file with metadata headers and a raw HTML file for the requested resource) to your network interface which hands it to your browser. Finally your browser displays the response as a web page.

Client-server model — How does it work?

Client (like a web broswer) makes a request to a server and waits for a response. Browsers are responsible for issuing HTTP requests and processing raw responses as web pages on your screen.

What is the role of HTTP as a protocol within the client server model? It provides uniformity to the way web resources are transferred between applications by providing a structure for messages and their data. It’s not concerned with how the messages are transported — it leaves that to underlying layers — mostly TCP/IP.

URLs (aka Uniform Resource Locators)

Components of a URL

1. Scheme: tells client how to access resource (ex: http, https)
2. Host: tells client where resource is located. translated to IP address by DNS. (ex: google.com)
3. Port (not required): designates which port number to listen for http requests. Only required if you want to use a port other than the default :80. (ex: http://localhost:300/profile)
4. Path (not required): tells client which local resource is being requested (ex: /home)
5. Query string/Parameters (not required): tells client some data to send to the server. starts with “?” and made up of query parameters formatted as name=value pairs and separated by “&”. Only used in GET requests. (ex. ?item=book&author=couser)

Path and Parameters are the components of a URL that allow you to pass information about the request to the server.

URL vs URI

URL is a subset of URI that includes the network location of the resource.

Scheme vs Protocol

Scheme is lowercase and specifies how to assign identifiers. Scheme identifies which protocol to be used. Protocol is uppercase and doesn’t have an impact on scheme.

How to construct a valid URL

• https:// (scheme)
• launchschool.com (host)
• /forum/search (path)
• ?utf8=✓&term=mastery (query)

Rules of query strings:

• there are max length limits to query strings
• name value pairs are visible in URL (not good for username password pairs)
• spaces and special characters are not allowed. allowed characters are alphanumeric characters and $-+.+’()”,

URL encoding

What is it? Writing special characters into a URL by replacing the characters with a % symbol followed by the hexidecimal ASCII code for the character. (ex: /tommy%20hilfiger.html)

When might URL encoding be used?

1. when the special character maybe be misinterpreted or modified by some systems (ex: %)
2. when the special character is reserved for another use in a URL (ex: ? or /)
3. when there is no corresponding character within the standard ASCII character set (ex: characters not in the roman alphabet)

HTTP and the Request/Response Cycle

What is the HTTP request response cycle?

HTTP is a text based protocol that uses requests and responses to structure and send messages over the internet. A user inputs a URL into a client. The client uses to URL to generate a request for a given resource, and send that information to a server using TCP/IP. The server receives the request, and uses the request’s information to generate a response containing the requested information (usually in the form of an html page). The server sends the response back to the client in the requested format, and the client renders the response as a web page.

What are HTTP requests?

Components of requests:

• Method: tells sever what action to perform on a resource. Identifies the kind of HTTP request you are sending. (ex: GET & POST)
• Path: tells server what resource to perform the method on
• HTTP version: identifies compatible syntax for the request/response.
• Host header: identifies the client.
• Other Headers (optional): colon-separated name-value pairs. allow client to send important metadata during request/response cycle. (ex: acceptable languages, user agent, type of connection preferred by client)
• HTTP Body (optional): contains the data that is being transmitted in an HTTP message.

Request types

GET — what is it and when should I use it? GET retrieves a resource from a server. This is the type of request you initiate when you type a URL into a browser address bar. It’s like a “read-only” request. Not used for forms unless its a search form (which doesn’t modify the resource).

POST — what is it and when should I use it? POST modifies a resource, sends data to server, or initiates action on server like submitting a form. (ex: $ curl -X POST “http://al-blackjack.herokuapp.com/new_player" -d “player_name=Albert” -m 30 -v). It does this through the HTTP body rather than the URL. Some advantages of a POST request are:

• It lets you side step the query string size limitation and lets you send larger data to server like images and videos
• It avoids exposing your private credentials in a URL when submitting sensitive data in a form

What’s the difference? GET retrieves and POST modifies.

What are HTTP responses?

Components of responses:

• Status code: three-digit number sent by server that signifies the status of the sender’s request (see below)
• Headers (optional): colon-separated name-value pairs. allow server to send important metadata during request/response cycle. (ex: type of encoding used on data, name of server, location to initiate new HTTP request, type of content contained in response)
• HTTP body (optional): raw response data sent back from server. usually html.

Status Codes

What are they? The three digit number sent by the server that signifies the status of the sender’s request.

What are some common types? (with examples)

• 200 OK — the request was handles successfully
• 302 Found — the requested resource has changed temporarily, but the browser knows this, and will automatically make a new request to the URL in the location response header (ex// entering the URL for a profile page might redirect you to a sign-in page through the location response header)
• 404 Not Found — the requested resource cannot be found (https://www.dropbox.com/awesome_file.jpg)
• 500 Internal Server Error — the server has encountered a generic error. could be a mis-configured server setting, a misplaced comma in the application code, or anything going wrong on the server side

“State” on the web

What does it mean for a protocol to be stateless? It means that each request-response cycle is independent of previous ones. The server does not save any information between requests. Some advantages of statelessness are that the protocol is resilient, distributed, and hard for others to control or manipulate. Unfortunately that also means it is difficult to build full-state applications on top of, and it’s difficult to secure. HTTP is an example of a stateless protocol. However you can simulate state by sending data as parameters through the URL, by using sessions, or by using cookies.

How do web applications use sessions to simulate state? Sessions occur when the server sends a unique session identifier token to the client. Afterwards, whenever that client makes an HTTP request to the server, it sends the token as well, and so the server can identify the client — and respond as if it remembered the page’s state. Step by step this looks like:

1. Client sends request to server
2. Server inspects request for session identifier and checks to see if it has expired
3. Server retrieves session data based on id and recreates application state (html from web request) from that session data
4. Server sends application state as response

What are cookies? Cookies are data about session information that are sent from the server and are stored by the client/browser during a request response cycle. After you access a website for the first time, the session info gets sent from the server and set in your browser cookie on your local computer. The next time you send a request, the client cookie and server session data are compared to identify the current session. Step by step it looks like this:

1. Client sends request to website for the first time
2. Server sens response with “set cookie” header
3. When client sends another request, the “cookie” hearder will be set to the value of the response’s “set cookie” header

What is AJAX? Asynchronous Javascript and XML. AJAX requests are sent to severs like normal HTTP requests and are handled by the server as usual. On the client side though, the browser doesn’t refresh when it processes the response. The response is processed by a callback function on the client side. This allows browsers to issue and process requests without a full page refresh.

Server-side vs client-side frameworks

• Server side frameworks produce html pages (which form the body of an HTTP response). They use templates and stored data to formulate the html response.
• Client-side frameworks recieve raw data in an HTTP response body, and then manipulate that data in application before rendering it.
• Applications within both types of frameworks use their own logic to interpret the path in the URL.

What’s up with servers?

• Web servers handle requests for static assets that don’t require data processing
• Application servers handle complicated requests that require logical data processing
• Data stores save data for later retrieval and processing by the application server.

Security

What are some security risks affecting HTTP?

HTTP is transferred in plain text, and doesn’t provide any means of secure message transfer. That leaves you risk to:

• Packet Sniffing/Session Hijacking — someone finding your session id and accessing your private data
• Cross-site scripting (XSS) — someone taking advantage of the ability to input HTML or JavaScript onto a website that gets displayed directly on the site. Without security measures, user input gets injected directly into page contents and then interpreted by the browser as HTML/javascript and executed. the code then lives on the site.

What are some measures to mitigate against security risks?

• Same Origin Policy — restricts some interactions between resources that originate from different origins to guard against session hijacking. The only allowed interactions are linking, redirects, form submission, and embedding. This prevents resources from being accessed by APIs like XMLHttpRequest or fetch. “Origin” means the combination of URL scheme, host, and port. http://mysite.com/doc1 has the same origin as http://mysite.com/doc2, but different scheme than https://mysite.com/doc2, a different host than http://myothersite.com/doc2, and a different port than http://mysite.com:4000/doc2.
• Resetting Sessions — prevents session hijacking by making sure users authenticate when entering a potentially sensitive area (ex: charging credit card, deleting account, etc)
• Setting session expiration time — prevents session hijacking by making users re-authenticate after a certain amount of time (chase website expiring after 30 min)
• Using HTTPS across the entire app — prevents session hijacking by encrypting every request-response cycle before transporting on the network. Encryption uses SLL/TLS with TCP at the Transport later. TLS is a cryptographic protocol that uses certificates to exchanges security keys before encrypting data. (more below)
• Sanitize user input — prevents xss by forbidding HTML and JavaScript in session input (only allowing something like markdown format) & eliminating markers of bad inputs like <script>.
• Escape user input data — prevents xss through character escaping so computer won’t interpret input as code. (ex: <p>Hello World!<\p> becomes <p>Hello World!<\p&gt)

What services does TLS provide?

1. Encryption — encoding message so it can only be read by someone authorized to decode the message (key encryption + handshake)
2. Authentication — verifying identity of a party in the message exchange (certificates)
3. Integrity — detecting if a message has been interfered with or faked (MAC code)

How does TLS provide encryption?

Symmetric Key Encryption: Makes up the bulk of TLS message exchange. Sender and receiver share a common encryption key.

Asymmetric Key Encryption: Uses a pair of keys — a public key and a private key. The public key encrypts, private key decrypts. It only works one way. “Hey you can send me secret messages with this public key! But nobody will know how I decrypt it cuz that’s private.”

TLS Handshakes

How a client and server exchange encryption keys. Occurs immediately after TCP handshake when using HTTPS, and results in an agreement of TLS version, cipher suite algorithms, and symmetric keys. Adds several rounds of latency to a message transfer.

• Client sends ClientHello message, which contains max version of TLS protocol and a list of cipher suites (set of cryptographic algorithms)
• Server responds with ServerHello which sets the compatible protocol version and cipher suite. Server also sends a certificate (shares public encryption key and provides means of ID for certificate provider) and a ServerHelloDone marker.
• Client responds by initiating key exchange based on the rules of the cipher suite. Involves using Server’s public key to send an encrypted message and generate the same symmetric key.
• Server sends changeCipherSpec and finish flags. They can now communicate with symmetric encryption.

Certificates

How do they work? The server sends its certificate, which includes its public key. The server creates a ‘signature’ in the form of some data encrypted with the server’s private key. The signature is transmitted in a message along with the original data from which the signature was created. Upon receipt of the message, the client decrypts the signature using the server’s public key and compares the decrypted data to the original version. If the two versions match then the encrypted version could only have been created by a party in possession of the private key.

Who authenticates certificates? Certificate authorities — a chain of trusted authorities who can sign certificates. There are a few root CAs, then more intermediate CAs which allows for extra security — if an intermediate CA is compromised that doesn’t affect every certificate signed by the root CA, so the root CA can issue new certificates to the intermediate branch to solve the problem. They ensure security by verifying that the server requesting authentication is who they say they are, and then they digitally sign the certificate.

Message Authentication Code (MAC)

A field that operates like a checksum to ensure messages weren’t manipulated in transit. It uses a hashing algorithm and a message digest to confirm the integrity of the message.

Evolution of HTTP

HTTP has evolved to adapt to a changing internet ecosystem.

Browser tech has improved dramatically since the advent of the internet. First dial-up arrived, then W3C and HTTP-WG frameworks for web tech standardization led to a need to standardize, then graphics pushed for an improvement in efficiency, etc.

• v0.9 one line and very simple GET + path
• v1 HEAD and POST methods were added to GET, path or absolute URI accepted as part of request line, HTTP version added, responses included status line, addition of metadata in the form of headers
• v1.1 more methods, connection re-use and pipelining (it was not efficient to keep closing connection after 1 http req-resp cycle)
• v2 multiplexing instead of pipelining, compression of headers
• v3 (next up) uses QUIC protocol to provide more efficient reliability and security

What can browser APIs provide?

Help with data synchronization (XHR, SSE, Websocket). There’s other web optimizations as well that allow browsers to predict user behavior by observing patterns (speculative), or triage fetching resources so that time consuming resources are loaded first (document aware).

What networking paradigms exist beyond client server?

• Peer to Peer (P2P) where each computer acts as a node. No need to set up a server or to communicate info through a central point (good for video calls), but hard to find other nodes, and can have security, performances, and scaling issues.
• webRTC, a collection of standard, protocols, and APIs that abstracts away complexities of finding P2P nodes, and allows the browser itself to acts as a P2P node.

What is the main limiter of networked application performance and how can I deal with it?

Latency is the biggest issue in networked application performances. You can reduce latency by reducing unnecessary resources, using data compression, reusing TCP connections, reducing number of host names, host external resources, pay for a faster DNS provider, or cache on the server side! (store recently accessed content in server, so it can be delivered more quickly the next time)