From c9a6734f39bb1f172f10ff01e09dc9d19ad10f47 Mon Sep 17 00:00:00 2001 From: Kamran Ahmed Date: Wed, 29 Jan 2020 11:28:20 +0500 Subject: [PATCH] Use on heading per page --- content/guides/design-patterns-for-humans.md | 2 +- content/guides/torrent-client.md | 32 ++++++++++---------- pages/privacy.js | 18 +++++------ 3 files changed, 26 insertions(+), 26 deletions(-) diff --git a/content/guides/design-patterns-for-humans.md b/content/guides/design-patterns-for-humans.md index d4618c2b7..b5598e70a 100644 --- a/content/guides/design-patterns-for-humans.md +++ b/content/guides/design-patterns-for-humans.md @@ -1595,7 +1595,7 @@ $stationList->removeStation(new RadioStation(89)); // Will remove station 89 ``` 👽 Mediator -======== +-------- Real world example > A general example would be when you talk to someone on your mobile phone, there is a network provider sitting between you and them and your conversation goes through it instead of being directly sent. In this case network provider is mediator. diff --git a/content/guides/torrent-client.md b/content/guides/torrent-client.md index 89667c545..9dd2553c8 100644 --- a/content/guides/torrent-client.md +++ b/content/guides/torrent-client.md @@ -6,14 +6,14 @@ The protocol evolved organically over the past 20 years, and various people and I'll be using a [Debian ISO](https://cdimage.debian.org/debian-cd/current/amd64/bt-cd/#indexlist) file as my guinea pig because it's big, but not huge, at 350MB. As a popular Linux distribution, there will be lots of fast and cooperative peers for us to connect to. And we'll avoid the legal and ethical issues related to downloading pirated content. -# Finding peers +## Finding peers Here’s a problem: we want to download a file with BitTorrent, but it’s a peer-to-peer protocol and we have no idea where to find peers to download it from. This is a lot like moving to a new city and trying to make friends—maybe we’ll hit up a local pub or a meetup group! Centralized locations like these are the big idea behind trackers, which are central servers that introduce peers to each other. They’re just web servers running over HTTP, and you can find Debian’s at http://bttracker.debian.org:6969/ ![illustration of a desktop computer and laptop sitting at a pub](/guides/torrent-client/trackers.png) Of course, these central servers are liable to get raided by the feds if they facilitate peers exchanging illegal content. You may remember reading about trackers like TorrentSpy, Popcorn Time, and KickassTorrents getting seized and shut down. New methods cut out the middleman by making even **peer discovery** a distributed process. We won't be implementing them, but if you're interested, some terms you can research are **DHT**, **PEX**, and **magnet links**. -## Parsing a .torrent file +### Parsing a .torrent file A .torrent file describes the contents of a torrentable file and information for connecting to a tracker. It's all we need in order to kickstart the process of downloading a torrent. Debian's .torrent file looks like this: ```markdown @@ -106,7 +106,7 @@ func (bto *bencodeTorrent) toTorrentFile() (*TorrentFile, error) { } ``` -## Retrieving peers from the tracker +### Retrieving peers from the tracker Now that we have information about the file and its tracker, let's talk to the tracker to **announce** our presence as a peer and to retrieve a list of other peers. We just need to make a GET request to the `announce` URL supplied in the .torrent file, with a few query parameters: ```go @@ -136,7 +136,7 @@ The important ones: ![a file with a name tag saying 'info_hash' and a person with a name tag 'peer_id'](/guides/torrent-client/info-hash-peer-id.png) -## Parsing the tracker response +### Parsing the tracker response We get back a bencoded response: ```markdown @@ -179,7 +179,7 @@ func Unmarshal(peersBin []byte) ([]Peer, error) { } ``` -# Downloading from peers +## Downloading from peers Now that we have a list of peers, it's time to connect with them and start downloading pieces! We can break down the process into a few steps. For each peer, we want to: 1. Start a TCP connection with the peer. This is like starting a phone call. @@ -196,7 +196,7 @@ if err != nil { I set a timeout so that I don't waste too much time on peers that aren't going to let me connect. For the most part, it's a pretty standard TCP connection. -## Complete the handshake +### Complete the handshake We've just set up a connection with a peer, but we want do a handshake to validate our assumptions that the peer * can communicate using the BitTorrent protocol @@ -250,14 +250,14 @@ func Read(r io.Reader) (*Handshake, error) { } ``` -## Send and receive messages +### Send and receive messages Once we've completed the initial handshake, we can send and receive **messages**. Well, not quite—if the other peer isn't ready to accept messages, we can't send any until they tell us they're ready. In this state, we're considered **choked** by the other peer. They'll send us an **unchoke** message to let us know that we can begin asking them for data. By default, we assume that we're choked until proven otherwise. Once we've been unchoked, we can then begin sending **requests** for pieces, and they can send us messages back containing pieces. !["A cartoon in which person 1 says 'hello I would like piece number—' and person 2 grabs him by the neck and says '00 00 00 01 00 (choke)'](/guides/torrent-client/choke.png) -### Interpreting messages +#### Interpreting messages A message has a length, an **ID** and a **payload**. On the wire, it looks like: ![A message with 4 byte for the length, 1 byte for ID, and an optional payload](/guides/torrent-client/message.png) @@ -333,7 +333,7 @@ func Read(r io.Reader) (*Message, error) { } ``` -### Bitfields +#### Bitfields One of the most interesting types of message is the **bitfield**, which is a data structure that peers use to efficiently encode which pieces they are able to send us. A bitfield looks like a byte array, and to check which pieces they have, we just need to look at the positions of the *bits* set to 1. You can think of it like the digital equivalent of a coffee shop loyalty card. We start with a blank card of all `0`, and flip bits to `1` to mark their positions as "stamped." ![a coffee shop loyalty card with eight slots, with stamps on the first four slots and a stamp on the second to last slot, represented as 11110010](/guides/torrent-client/bitfield.png) @@ -359,10 +359,10 @@ func (bf Bitfield) SetPiece(index int) { } ``` -## Putting it all together +### Putting it all together We now have all the tools we need to download a torrent: we have a list of peers obtained from the tracker, and we can communicate with them by dialing a TCP connection, initiating a handshake, and sending and receiving messages. Our last big problems are handling the **concurrency** involved in talking to multiple peers at once, and managing the **state** of our peers as we interact with them. These are both classically Hard problems. -### Managing concurrency: channels as queues +#### Managing concurrency: channels as queues In Go, we [share memory by communicating](https://blog.golang.org/share-memory-by-communicating), and we can think of a Go channel as a cheap thread-safe queue. We'll set up two channels to synchronize our concurrent workers: one for dishing out work (pieces to download) between peers, and another for collecting downloaded pieces. As downloaded pieces come in through the results channel, we can copy them into a buffer to start assembling our complete file. @@ -437,7 +437,7 @@ func (t *Torrent) startDownloadWorker(peer peers.Peer, workQueue chan *pieceWork } ``` -### Managing state +#### Managing state We'll keep track of each peer in a struct, and modify that struct as we read messages. It'll include data like how much we've downloaded from the peer, how much we've requested from them, and whether we're choked. If we wanted to scale this further, we could formalize this as a finite state machine. But a struct and a switch are good enough for now. ```go @@ -469,12 +469,12 @@ func (state *pieceProgress) readMessage() error { } ``` -### Time to make requests! +#### Time to make requests! Files, pieces, and piece hashes aren't the full story—we can go further by breaking down pieces into **blocks**. A block is a part of a piece, and we can fully define a block by the **index** of the piece it's part of, its byte **offset** within the piece, and its **length**. When we make requests for data from peers, we are actually requesting *blocks*. A block is usually 16KB large, meaning that a single 256 KB piece might actually require 16 requests. A peer is supposed to sever the connection if they receive a request for a block larger than 16KB. However, based on my experience, they're often perfectly happy to satisfy requests up to 128KB. I only got moderate gains in overall speed with larger block sizes, so it's probably better to stick with the spec. -### Pipelining +#### Pipelining Network round-trips are expensive, and requesting each block one by one will absolutely tank the performance of our download. Therefore, it's important to **pipeline** our requests such that we keep up a constant pressure of some number of unfulfilled requests. This can increase the throughput of our connection by an order of magnitude. ![Two email threads simulating peer connections. The thread on the left shows a request followed by a reply, repeated three times. The thread on the left sends three requests, and receives three replies in quick succession.](/guides/torrent-client/pipelining.png) @@ -529,7 +529,7 @@ func attemptDownloadPiece(c *client.Client, pw *pieceWork) ([]byte, error) { } ``` -### main.go +#### main.go This is a short one. We're almost there. ```go @@ -560,5 +560,5 @@ func main() { -# This isn't the full story +## This isn't the full story For brevity, I included only a few of the important snippets of code. Notably, I left out all the glue code, parsing, unit tests, and the boring parts that build character. View my [full implementation](https://github.com/veggiedefender/torrent-client) if you're interested. diff --git a/pages/privacy.js b/pages/privacy.js index 2d0961c5a..78fe49c32 100644 --- a/pages/privacy.js +++ b/pages/privacy.js @@ -15,7 +15,7 @@ const Privacy = () => (

By using or accessing the Services in any manner, you acknowledge that you accept the practices and policies outlined in this Privacy Policy, and you hereby consent that we will collect, use, and share your information in the following ways. Remember that your use of roadmap.sh’s Services is at all times subject to the Terms of Use, which incorporates this Privacy Policy. Any terms we use in this Policy without defining them have the definitions given to them in the Terms of Use.

-

What does this Privacy Policy cover?

+

What does this Privacy Policy cover?

This Privacy Policy covers our treatment of personally identifiable information ("Personal Information") that we gather when you are accessing or using our Services, but not to the practices of companies we don’t own or control, or people that we don’t manage. We gather various types of Personal Information from our users, as explained in more detail below, and we use this Personal Information internally in connection with our Services, including to personalize, provide, and improve our services, to allow you to set up a user account and profile, to contact you and allow other users to contact you, to fulfill your requests for certain products and services, and to analyze how you use the Services. In certain cases, we may also share some Personal Information with third parties, but only as described below.

@@ -23,13 +23,13 @@ const Privacy = () => ( age 13, we will delete that information as quickly as possible. If you believe that a child under 13 may have provided us personal information, please contact us at kamran@roadmap.sh.

-

Will roadmap.sh ever change this Privacy Policy?

+

Will roadmap.sh ever change this Privacy Policy?

We’re constantly trying to improve our Services, so we may need to change this Privacy Policy from time to time as well, but we will alert you to changes by updating the services on the website, placing a notice on the Services, by sending you an email, and/or by some other means. Please note that if you’ve opted not to receive legal notice emails from us (or you haven’t provided us with your email address), those legal notices will still govern your use of the Services, and you are still responsible for reading and understanding them. If you use the Services after any changes to the Privacy Policy have been posted, that means you agree to all of the changes. Use of information we collect now is subject to the Privacy Policy in effect at the time such information is used or collected.

-

What Information does roadmap.sh Collect?

+

What Information does roadmap.sh Collect?

Information You Provide to Us:

We receive and store any information you knowingly provide to us. For example, through the registration process and/or through your account settings, we may collect Personal Information such as your name, title, email address, phone number, and third-party account credentials (for example, your log-in credentials for Twitter or @@ -38,7 +38,7 @@ const Privacy = () => (

We may communicate with you if you’ve provided us the means to do so. For example, if you’ve given us your email address, we may send you promotional email offers on behalf of other businesses, or email you about your use of the Services. Also, we may receive a confirmation when you open an email from us. This confirmation helps us make our communications with you more interesting and improve our services. If you do not want to receive communications from us, please email us at kamran@roadmap.sh.

-

Information Collected Automatically

+

Information Collected Automatically

Whenever you interact with our Services, we automatically receive and record information on our server logs from your browser or device, which may include your IP address, geolocation data, device identification, “cookie” information, the type of browser and/or device you’re using to access our Services, and the page or feature you requested. “Cookies” are identifiers we transfer to your browser or device that allow us to recognize your browser or device and tell us how and when pages and features in our Services are visited and by how many people. You may be able to change the preferences on your browser or device to prevent or limit your device’s @@ -48,7 +48,7 @@ const Privacy = () => ( many users as possible.

-

Will roadmap.sh Share Any of the Personal Information it Receives?

+

Will roadmap.sh Share Any of the Personal Information it Receives?

We may share your Personal Information with third parties as described in this section:

Information that’s no longer personally identifiable. We may anonymize your Personal Information so that you are not individually identified, and provide that information to our partners. We may also provide aggregate usage information to our partners, who may use such information to understand how often and in what ways people @@ -77,12 +77,12 @@ const Privacy = () => ( users, or others.

-

Is Personal Information about me secure?

+

Is Personal Information about me secure?

Your account is protected by a password for your privacy and security. If you access your account via a third party site or service, you may have additional or different sign-on protections via that third party site or service. You must prevent unauthorized access to your account and Personal Information by selecting and protecting your password and/or other sign-on mechanism appropriately and limiting access to your computer or device and browser by signing off after you have finished accessing your account. We endeavor to protect the privacy of your account and other Personal Information we hold in our records, but unfortunately, we cannot guarantee complete security. Unauthorized entry or use, hardware or software failure, and other factors, may compromise the security of user information at any time.

-

What Personal Information can I access?

+

What Personal Information can I access?

Through your account settings, you may access, and, in some cases, edit or delete the following information you’ve provided to us: