BitTorrent: Pieces, Peer Discovery and Swarm Limits

BitTorrent: distribute the upload work

BitTorrent lets recipients exchange verified pieces of a file with one another. A peer can upload completed pieces before it has the whole file, so distribution capacity can grow with the audience. It still needs reachable peers, useful pieces, and enough aggregate upload bandwidth.

The original protocol is specified in BEP 3. It describes metainfo, tracker interaction, the peer wire protocol, and piece exchange. Protocol facts below should be distinguished from the illustrative workload calculations. BitTorrent BEP 3.

A distribution budget

Suppose a publisher distributes a 1 GiB file to 100 recipients. Sending 100 complete copies requires 100 GiB of delivered data. A single 1 Gbps publisher link would take at least about 859 seconds—14.3 minutes—if it sustained that rate without overhead or competing traffic.

Now suppose every recipient is willing and able to upload at 10 Mbps. Their eventual combined upload contribution is 1 Gbps. This can reduce the publisher bottleneck, but it does not halve completion time automatically. Peers begin without useful pieces, connections have overhead, upload capacity varies, and the slowest recipient's download path can dominate its own completion time.

The relevant resource is useful verified data delivered, not merely the number of peers listed by a tracker. Ten thousand unreachable peers offer no bandwidth.

Metadata identifies the content

For an illustrative v1 torrent, a 1 GiB payload divided into 1 MiB pieces produces 1,024 piece hashes. At 20 bytes per SHA-1 digest, that list occupies 20 KiB before other metadata. Piece size affects metadata overhead, scheduling granularity, and how much data must be redownloaded when verification fails.

In v1, the info hash is calculated over the exact bencoded info dictionary bytes. Parsing and carelessly re-encoding a noncanonical input can change those bytes and the resulting identity. Use a conforming implementation rather than hashing an arbitrary JSON representation.

BEP 52 defines BitTorrent v2, including SHA-256-based Merkle tree structures and changed metadata rules. Do not apply a v1 SHA-1 description to every current torrent; hybrid torrents add further interoperability details. BitTorrent v2, BEP 52.

A valid piece hash establishes agreement with the torrent's metadata. It does not establish that the publisher is trustworthy, that the file is harmless, or that distribution is authorized. Obtain the torrent identity from a source appropriate to the content's trust requirements.

Discovery finds peers, not file bytes

A tracker can return candidate peer addresses for a torrent. A distributed hash table is another discovery mechanism; BEP 5 specifies peer discovery keyed by an info hash using a Kademlia-style DHT. The DHT does not contain the payload itself. BitTorrent DHT, BEP 5.

A magnet link identifies what is sought and can carry discovery information, but clients still need peers and metadata before they can assemble the files. NAT, firewalls, and churn affect which candidate peers can actually connect.

Follow a piece through the swarm

A peer learns what its neighbors hold, chooses wanted pieces, and requests blocks within those pieces. When enough blocks arrive, it verifies the completed piece before marking it available and announcing it. A corrupted piece is discarded and requested again; data should not be trusted merely because it arrived through a valid connection.

A rarest-first strategy prioritizes pieces with fewer known copies, improving the chance the swarm retains every part. During startup, a client may choose differently to acquire something useful to trade quickly. Near completion, carefully bounded duplicate requests can reduce waiting on one slow peer, with cancellation when one copy arrives. For an implementation-specific discussion of selection strategies and their trade-offs, see the libtorrent piece-picker documentation.

Choking controls which peers may receive uploads; optimistic unchoking explores new partners. These mechanisms encourage useful exchange, but actual clients have different policies and operating conditions. “Every peer uploads half its download speed” is not a protocol guarantee.

Availability is piece-level

Imagine all peers have 99% of a release, but the only peer holding the final piece disconnects. The aggregate swarm looks almost complete, yet nobody can finish. A publisher that requires reliable availability should retain complete seeds and monitor piece availability, not just the number of participants.

For a private release, make discovery, access control, and retention explicit. Public peer discovery can reveal participation and network addresses. Hashes verify bytes; they do not substitute for authentication or confidentiality.

A useful experiment distributes a small authorized test file across controlled peers, removes a seed, throttles uploads, and injects a corrupted piece. Record completion time, rejected pieces, useful upload throughput, and time waiting for rare pieces. The experiment can explain whether peer distribution suits a fleet release or a large public download. Small files, tiny audiences, and strict startup latency may be better served by ordinary HTTP delivery or a CDN.