diff --git a/src/roadmaps/system-design/content/115-communication/100-http.md b/src/roadmaps/system-design/content/115-communication/100-http.md index 5edc4bab2..0cd773329 100644 --- a/src/roadmaps/system-design/content/115-communication/100-http.md +++ b/src/roadmaps/system-design/content/115-communication/100-http.md @@ -2,7 +2,20 @@ HTTP is a method for encoding and transporting data between a client and a server. It is a request/response protocol: clients issue requests and servers issue responses with relevant content and completion status info about the request. HTTP is self-contained, allowing requests and responses to flow through many intermediate routers and servers that perform load balancing, caching, encryption, and compression. -To learn more, visit the following links: +A basic HTTP request consists of a verb (method) and a resource (endpoint). Below are common HTTP verbs: + +``` +Verb | Description | Idempotent* | Safe | Cacheable | +-------|-------------------------------|-------------|------|-----------------------------------------| +GET | Reads a resource | Yes | Yes | Yes | +POST | Creates a resource or trigger | No | No | Yes if response contains freshness info | +PUT | Creates or replace a resource | Yes | No | No | +PATCH | Partially updates a resource | No | No | Yes if response contains freshness info | +DELETE | Deletes a resource | Yes | No | No | + +``` + +HTTP is an application layer protocol relying on lower-level protocols such as TCP and UDP. - [What Is HTTP?](https://www.nginx.com/resources/glossary/http/) - [What is the difference between HTTP protocol and TCP protocol?](https://www.quora.com/What-is-the-difference-between-HTTP-protocol-and-TCP-protocol) \ No newline at end of file diff --git a/src/roadmaps/system-design/content/115-communication/101-tcp.md b/src/roadmaps/system-design/content/115-communication/101-tcp.md index b486a1498..35b427c6e 100644 --- a/src/roadmaps/system-design/content/115-communication/101-tcp.md +++ b/src/roadmaps/system-design/content/115-communication/101-tcp.md @@ -1,13 +1,28 @@ # TCP -TCP is a connection-oriented protocol over an IP network. Connection is established and terminated using a handshake. All packets sent are guaranteed to reach the destination in the original order and without corruption through: +TCP is a connection-oriented protocol over an [IP network](https://en.wikipedia.org/wiki/Internet_Protocol). Connection is established and terminated using a [handshake](https://en.wikipedia.org/wiki/Handshaking). All packets sent are guaranteed to reach the destination in the original order and without corruption through: -- Sequence numbers and checksum fields for each packet -- Acknowledgement packets and automatic retransmission +- Sequence numbers and [checksum fields](https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Checksum_computation) for each packet +- [Acknowledgement](https://en.wikipedia.org/wiki/Acknowledgement_(data_networks)) packets and automatic retransmission -If the sender does not receive a correct response, it will resend the packets. If there are multiple timeouts, the connection is dropped. TCP also implements flow control and congestion control. These guarantees cause delays and generally result in less efficient transmission than UDP. +If the sender does not receive a correct response, it will resend the packets. If there are multiple timeouts, the connection is dropped. TCP also implements [flow control](https://en.wikipedia.org/wiki/Flow_control_(data)) and congestion control. These guarantees cause delays and generally result in less efficient transmission than UDP. + +To ensure high throughput, web servers can keep a large number of TCP connections open, resulting in high memory usage. It can be expensive to have a large number of open connections between web server threads and say, a [memcached server](https://memcached.org/). [Connection pooling](https://en.wikipedia.org/wiki/Connection_pool) can help in addition to switching to UDP where applicable. + +TCP is useful for applications that require high reliability but are less time critical. Some examples include web servers, database info, SMTP, FTP, and SSH. + +Use TCP over UDP when: + +- You need all of the data to arrive intact +- You want to automatically make a best estimate use of the network throughput To learn more, visit the following links: - [What Is TCP?](https://github.com/donnemartin/system-design-primer#TCP) -- [What is the difference between HTTP protocol and TCP protocol?](https://www.quora.com/What-is-the-difference-between-HTTP-protocol-and-TCP-protocol) \ No newline at end of file +- [What is the difference between HTTP protocol and TCP protocol?](https://www.quora.com/What-is-the-difference-between-HTTP-protocol-and-TCP-protocol) +- [Networking for game programming](http://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/) +- [Key differences between TCP and UDP protocols](http://www.cyberciti.biz/faq/key-differences-between-tcp-and-udp-protocols/) +- [Difference between TCP and UDP](http://stackoverflow.com/questions/5970383/difference-between-tcp-and-udp) +- [Transmission control protocol](https://en.wikipedia.org/wiki/Transmission_Control_Protocol) +- [User datagram protocol](https://en.wikipedia.org/wiki/User_Datagram_Protocol) +- [Scaling memcache at Facebook](http://www.cs.bu.edu/~jappavoo/jappavoo.github.com/451/papers/memcache-fb.pdf) \ No newline at end of file diff --git a/src/roadmaps/system-design/content/115-communication/102-udp.md b/src/roadmaps/system-design/content/115-communication/102-udp.md index f01e13681..6b9c19716 100644 --- a/src/roadmaps/system-design/content/115-communication/102-udp.md +++ b/src/roadmaps/system-design/content/115-communication/102-udp.md @@ -14,5 +14,9 @@ Use UDP over TCP when: To learn more, visit the following link: -- [What Is UDP?](https://github.com/donnemartin/system-design-primer#UDP) -- [Difference between TCP and UDP?](https://stackoverflow.com/questions/5970383/difference-between-tcp-and-udp) \ No newline at end of file +- [Networking for game programming](http://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/) +- [Key differences between TCP and UDP protocols](http://www.cyberciti.biz/faq/key-differences-between-tcp-and-udp-protocols/) +- [Difference between TCP and UDP](http://stackoverflow.com/questions/5970383/difference-between-tcp-and-udp) +- [Transmission control protocol](https://en.wikipedia.org/wiki/Transmission_Control_Protocol) +- [User datagram protocol](https://en.wikipedia.org/wiki/User_Datagram_Protocol) +- [Scaling memcache at Facebook](http://www.cs.bu.edu/~jappavoo/jappavoo.github.com/451/papers/memcache-fb.pdf) \ No newline at end of file diff --git a/src/roadmaps/system-design/content/115-communication/103-rpc.md b/src/roadmaps/system-design/content/115-communication/103-rpc.md index 205ca03e0..26818f34e 100644 --- a/src/roadmaps/system-design/content/115-communication/103-rpc.md +++ b/src/roadmaps/system-design/content/115-communication/103-rpc.md @@ -1,6 +1,36 @@ # RPC -In an RPC, a client causes a procedure to execute on a different address space, usually a remote server. The procedure is coded as if it were a local procedure call, abstracting away the details of how to communicate with the server from the client program. Remote calls are usually slower and less reliable than local calls so it is helpful to distinguish RPC calls from local calls. Popular RPC frameworks include Protobuf, Thrift, and Avro. +In an RPC, a client causes a procedure to execute on a different address space, usually a remote server. The procedure is coded as if it were a local procedure call, abstracting away the details of how to communicate with the server from the client program. Remote calls are usually slower and less reliable than local calls so it is helpful to distinguish RPC calls from local calls. Popular RPC frameworks include [Protobuf](https://developers.google.com/protocol-buffers/), [Thrift](https://thrift.apache.org/), and [Avro](https://avro.apache.org/docs/current/). + +RPC is a request-response protocol: + +- Client program - Calls the client stub procedure. The parameters are pushed onto the stack like a local procedure call. +- Client stub procedure - Marshals (packs) procedure id and arguments into a request message. +- Client communication module - OS sends the message from the client to the server. +- Server communication module - OS passes the incoming packets to the server stub procedure. +- Server stub procedure - Unmarshalls the results, calls the server procedure matching the procedure id and passes the given arguments. +- The server response repeats the steps above in reverse order. + +Sample RPC calls: + +``` +GET /someoperation?data=anId + +POST /anotheroperation +{ + "data":"anId"; + "anotherdata": "another value" +} +``` + +RPC is focused on exposing behaviors. RPCs are often used for performance reasons with internal communications, as you can hand-craft native calls to better fit your use cases. + +## Disadvantage of RPC + +- RPC clients become tightly coupled to the service implementation. +- A new API must be defined for every new operation or use case. +- It can be difficult to debug RPC. +- You might not be able to leverage existing technologies out of the box. For example, it might require additional effort to ensure [RPC calls are properly cached](http://etherealbits.com/2012/12/debunking-the-myths-of-rpc-rest/) on caching servers such as [Squid](http://www.squid-cache.org/). To learn more, visit the following links: diff --git a/src/roadmaps/system-design/content/115-communication/104-rest.md b/src/roadmaps/system-design/content/115-communication/104-rest.md index 99fc65ac9..1e47d1140 100644 --- a/src/roadmaps/system-design/content/115-communication/104-rest.md +++ b/src/roadmaps/system-design/content/115-communication/104-rest.md @@ -9,6 +9,8 @@ There are four qualities of a RESTful interface: - Self-descriptive error message (status response in HTTP) - Use status codes, don't reinvent the wheel. - HATEOAS (HTML interface for HTTP) - your web service should be fully accessible in a browser. +REST is focused on exposing data. It minimizes the coupling between client/server and is often used for public HTTP APIs. REST uses a more generic and uniform method of exposing resources through URIs, representation through headers, and actions through verbs such as GET, POST, PUT, DELETE, and PATCH. Being stateless, REST is great for horizontal scaling and partitioning. + To learn more, visit the following links: - [What Is REST?](https://github.com/donnemartin/system-design-primer#REST) diff --git a/src/roadmaps/system-design/content/115-communication/index.md b/src/roadmaps/system-design/content/115-communication/index.md index 4aec47310..addedcd0b 100644 --- a/src/roadmaps/system-design/content/115-communication/index.md +++ b/src/roadmaps/system-design/content/115-communication/index.md @@ -1,28 +1,3 @@ # Communication -## Hypertext transfer protocol (HTTP) -HTTP is a method for encoding and transporting data between a client and a server. It is a request/response protocol: clients issue requests and servers issue responses with relevant content and completion status info about the request. HTTP is self-contained, allowing requests and responses to flow through many intermediate routers and servers that perform load balancing, caching, encryption, and compression. - - -## Transmission control protocol (TCP) -TCP is a connection-oriented protocol over an IP network. Connection is established and terminated using a handshake. All packets sent are guaranteed to reach the destination in the original order and without corruption through: - -- Sequence numbers and checksum fields for each packet -- Acknowledgement packets and automatic retransmission - - -## User datagram protocol (UDP) -UDP is connectionless. Datagrams (analogous to packets) are guaranteed only at the datagram level. Datagrams might reach their destination out of order or not at all. UDP does not support congestion control. Without the guarantees that TCP support, UDP is generally more efficient. - -UDP can broadcast, sending datagrams to all devices on the subnet. This is useful with DHCP because the client has not yet received an IP address, thus preventing a way for TCP to stream without the IP address. - - -## Remote procedure call (RPC) -In an RPC, a client causes a procedure to execute on a different address space, usually a remote server. The procedure is coded as if it were a local procedure call, abstracting away the details of how to communicate with the server from the client program. Remote calls are usually slower and less reliable than local calls so it is helpful to distinguish RPC calls from local calls. Popular RPC frameworks include Protobuf, Thrift, and Avro. - -## Representational state transfer (REST) -REST is an architectural style enforcing a client/server model where the client acts on a set of resources managed by the server. The server provides a representation of resources and actions that can either manipulate or get a new representation of resources. All communication must be stateless and cacheable. - -To learn more, visit the following links: - -- [Getting started with Communication](https://github.com/donnemartin/system-design-primer) \ No newline at end of file +Network protocols are a key part of systems today, as no system can exist in isolation - they all need to communicate with each other. You should learn about the networking protocols such as HTTP, TCP, UDP. Also, learn about the architectural styles such as RPC, REST, GraphQL and gRPC. \ No newline at end of file