Bt   java bittorrent done right

Release 1.10

Release 1.10 was published to Maven Central on December 5th, 2021.

It’s been almost two years since 1.9 so this release is full to the brim with bug fixes, stability, performance and code improvements. It also brings some new features, most notably .torrent builder API and download priorities for inidividual files.

What really makes this release special though is that the absolute majority of work has been done by contributors. I’ve barely written a handful of lines of code and for the most part just participated in PR reviews and discussions and kept track of the work being done in the release notes doc.

To put it into perspective, here are some numbers:

  • in 1.9 there were 502 .java files containing 29400 lines of code
  • in 1.10 there are 523 .java files containing 31300 lines of code
  • since 1.9 there were more than 100 commits and over 200 changed files with about 7000 additions and 3500 deletions
  • 47 different units of work (features, fixes, improvements)

That’s an amazing effort by all standards, and I’m really thankful to all of you.

And so, here’s the full list of what went into 1.10:

New Features:

  • API to create torrent metainfo (.torrent files)
  • Update Guice to 5.0.1 for Java 9 support
  • Support callback functions for when
    • Torrent files complete downloading
    • The entire torrent completes downloading
  • ChooseFilesStage now allows SKIPPED, NORMAL_PRIORITY, and HIGH_PRIORITY for downloading
    • File download priority can be mutated while the torrent is downloaded. Skipping or unskipping files while the torrent is downloading is not supported.
  • Add config switch to completely disable MSE negotiation
  • Add config parameter for the time interval between reads in MSE negotiation
  • Add config parameter for the max number of simultaneously assigned pieces for a single peer
  • Add config parameter for the IP address to send to the tracker (for remote peers to use). Useful when behind a NAT. Only supported for http trackers currently.
  • Add runtime builder methods for disabling PEX and LSD extensions

Bug Fixes/Improvements:

  • Allow announce key to be missing in the torrent dictionary #42
  • bt.data.digest.JavaSecurityDigester performance improvement #133
  • Fix NullPointerException in SocketChannelHandler.flush()
  • Do not throw “Unexpectedly interrupted” exception when message loop is shutdown
  • Fix missing breaks in MetadataFetchStage which could led to hang-ups
  • Indicate the listening TCP port in thread names
  • Fix ConcurrentModificationException in LocalServiceDiscoveryService
  • Fix memory leak in BtRuntime by making sure that shutdown hook is removed after shutdown is complete
  • Fix memory leak in event bus mechanism by making sure that torrent listeners are removed when torrent is stopped
  • Ensure that torrent descriptor is inactivated when BtClient.stop() is invoked
  • Multiple minor improvements to reduce memory allocations
  • More descriptive error message when there’s insufficient room in buffer to read an integer
  • Implement LRU cache for open file handles (fixes issues with torrents with many files)
  • Fix NullPointerException in AdhocTorrentRegistry.unregister(TorrentId)
  • Updated the file I/O APIs to use FileChannel read/write so that multiple threads can concurrently read/write to different sections of an open file
  • Add API to forcefully flush torrent data to persistent storage
  • BTInteger uses Integer, Long, or BigInteger depending upon the number for space/performance savings
  • Download stage now uses a latch, rather than sleeping, to determine when the download is finished to go onto the seeding stage
  • Eliminate a 1 second wait for the initial torrent download by updating interested peers immediately if there are currently zero peers that the client is interested in
  • Torrent chunk verification now uses Stream APIs with a custom ForkJoinPool (if a desired amount of parallelism is requested) rather than an executor
  • DefaultDataDescriptor.filesForPieces now uses a List for indices rather than a map
  • LocalBitfield now uses a Copy On Write (COW) strategy for skipped pieces rathen than locking
  • PeerBitfields are read with BitSet.valueOf() rather than setting each Bit individually
  • Irrelevant pieces are filtered out of a bitset before they reach the piece selectors
  • The rarest piece selector now uses bucket sort based on max peer count rather than a MinHeap
  • The randomized rarest piece selector uses a precomputed random shuffle order to reduce the overhead of random piece shuffling
  • Fix double release issue in DefaultBorrowedBuffer, which eliminates the harmless exception “Buffer is locked and can’t be released” that occured sometimes
  • Fix intermittent NullPointerException in DHT shutdown hook
  • Lower logging level to debug on peer disconnects
  • Eliminate possibility of a dead lock in peer connection handling code #170
  • New configuration option for adjusting the amount of memory used for hashing (verification) of data
  • Fix a dead-lock in SocketChannelHandler #183
  • Runtime does not terminate when torrent has been downloaded #167
  • Peers received from the initial tracker connection are now used rather than discarded
  • Bt no longer makes two calls to the tracker on startup - one to announce the startup and the other to get peers.
  • Bt now respects tracker minimum announce intervals
  • Bt now does not send a completed event to the tracker if the torrent was already completed before it connected to the tracker
  • The first tracker announce is synchronously waited for - this decreases the time to connect to the first peers returned from the tracker.
  • Fix bug that prevents plain text clients from connecting when MSE encryption is enabled

Release 1.9

Release 1.9 was published to Maven Central on December 14th, 2019.

Major points of this release are:

  • improving throughput and cutting down on memory usage by implementing so-called “direct” I/O and better block request pipelining
  • improving peer lookup and connectivity by figuring out which connections are duplicate and eliminating those
  • allowing to create private DHTs within non-public networks (previously all nodes had to have public Internet address)

Bug Fixes/Improvements:

  • onTorrentStarted called twice #117
  • Add new onMetadataAvailable event (fired after torrent’s metadata has been fetched)
  • Allow cookie to be missing in the LSD announce message
  • Make sure outgoing messages are never dropped
  • Continuous piece assignments (better request pipelining)
  • Direct I/O
  • Add configuration option for the number of outstanding requests
  • Add configuration option for send/receive buffer size
  • Add CLI parameter for DHT port
  • Allow bogon addresses, when using private DHT
  • Reliably eliminate duplicate peer connections and self connections

Release 1.8

Release 1.8 was published to Maven Central on June 14th, 2019.

New Features:

  • UPnP port mapping #80
  • Generate OSGi manifests
  • Add bt.data.DataDescriptor.getReader() method for convenient stream-like access to torrent’s data

Bug Fixes/Improvements:

  • Outbound data silently discarded at high outgoing rates #94
  • Not possible to register consumers for a particular message type from more than one module #100
  • Support Base32-encoded info hashes in magnet links

Release 1.7

Release 1.7 was published to Maven Central on February 26th, 2018.

Changes/New Features:

  • Support for file selection (aka partial downloads)

Bug Fixes/Improvements:

  • Avoid creation of unnessary empty dirs when reading from a FileSystemStorageUnit that maps to an absent file
  • BEncoder: sort dictionary keys as raw byte sequences, not alphanumerical strings #50
  • Randomized rarest-first selector behaves like a sequential selector when peers are seeds #53
  • Empty files should not prevent successful verification of torrent’s data
  • NPE in DefaultChannelPipeline when there are unprocessed leftovers from MSE handshake #57
  • Incorrect behavior when the same peer participates in more than one torrent #67

Release 1.6

Release 1.6 was published to Maven Central on January 27th, 2018.

Official BEPs:

Bug Fixes/Improvements:

  • PeerTracker example does not work on Windows
  • Allow to selectively enable only a subset of standard extensions, like PEX and LSD
  • Re-use native memory messaging buffers between different peer connections
  • Check the allowed crypto key size and disable MSE if insufficient #24