Ошибка unexpected end of file in bencoded string торрент

О проблемах

Главная » Новости трекера » Работа трекера

Главная » Новости трекера » Работа трекера

So this happens with payload data going from uTorrent -> qBittorrent but not the other way around?

I cannot say whether qBitTorrent -> uTorrent would also result in packet resends (which implies packet loss)…because I lack the tools to check in that direction. I am relying on uTorrent’s graphs and it has no «knowledge» of what/which packets sent by qBitTorrent that get lost…at least as far as I can tell.

Where qBittorrent is essentially only sending ACKs and HAVE’s and other small protocol messages but no actual payload?

Yes to there being no actual data payload (since it’s a peer downloading from a lone uTorrent seed.)
…but no to that being the only packet types that qBitTorrent is sending.

Depending on how the first connection occurs, there can be a magnet link request based on hash URL.

If other features are enabled, the qBT peer would also be sending Peer EXchange packets, possibly DHT packets, and Local Peer/Seed Discovery packets…this is why I try to run such tests with as many things disabled as possible.
There is the self-announcing of alternate IP+port paths, (often IPv6 address if connected via IPv4…sometimes inadvertently bypassing a proxy or VPN that the user intended to be the only route due to not binding to the proxy/VPN network interface) which can happen even if PEX, LSD, and DHT are all disabled…but resembles PEX traffic.

There’s also BitTorrent flags, which probably get communicated directly or indirectly via packets.
Packets related to flag changes are sent:
https://superuser.com/questions/415185/what-do-flags-and-reqs-mean-in-utorrent
«Regarding the Reqs column: It’s not pieces, it’s how many (16KB) chunks that is active in the request queue currently.»

Reqs are for 16 KB chunks, but data packets are overwhelmingly limited to 1000-1500 bytes payload size.
So a single 16 KB chunk is split between multiple data packets, with the last one likely being a runt in size.

Currently libtorrent holds back sending undersized (<mtu) uTP packets until all other previously sent packets have been ACK’d

What if…qBT is sending malformed Reqs, or uTorrent’s ACK reply to those lists is partially ignored by qBT?
There could also be a problem with aggregate ACKs, which might only become an issue at higher upload speeds.

With a high enough latency between peers I can see why an extra RTT would be seen as potential packet loss and triggering a resend.

High latency isn’t needed to trigger these problems, although that can make speeds much slower.
A good hard-wired LAN can get pingtimes so low that they typically round down to 0 ms.
I’ve tested using that as well as internet connections (both IPv4 and/or IPv6).

But due to ease-of-testing I’ve mostly used 2 BT clients on the same PC connecting to each other via 127.0.0.1 local loopback.
For that, latency can be infinitesimal even compared to a good hard-wired LAN.
Yet I’m still seeing packet resends by uTorrent even there.

Also to eliminate 1 other bottleneck which might cause tiny amounts of latency, I’m using a single, small 1-4 GB ramdrive (but different folders for each BT client) to read from/write to for the torrent data.
This limits max torrent size I can use…but a 1 GB torrent with ~1 MB piece size uploading via uTP at 1 MB/sec still takes over 1/4 hour if I bother to let it run to completion.
Plenty of time to see ramp-up, cyclic speed issues, and steady-state rates.
I’ve tested using HDDs as well, but get the same results…unless the torrent’s piece size is 16 KB, then it depends on whether I’m using sequential downloading because HDDs are absolutely terrible for random small reads/writes.
For additional fun, I’ve even experimented with high file fragmentation on nearly full HDDs.

This would only happen if libtorrent isn’t sending a stream of payload data as then packets would tend to no longer be undersized and would send immediately.

The last part of a 16 KB chunk and individual pieces that straddle 2 or more files (where 1 ends and the next begins, or a couple tiny readme’s placed before a gigantic video file to badly misalign file clusters from torrent pieces…especially on a RAID NAS) might also sometimes cause undersized packets.

So I’ve tried out some more things that will hopefully help find the issue. Firstly maybe some context will help.

I am using Jackett as an indexer for my sites. Jackett generates and xml ‘torznab’ feed which is where I am pulling information about my torrents (including the download url which will be in the format https://<host>/jackett/dl/<indexer>/?jackett_apikey=<key>$path=<encoded>&file=<filename>.
When this URL is added to deluge manually through the webui, the torrent is recognised, the file information is pulled, and I am able to start downloading.

The above method is also how Sonarr adds torrents to deluge, except instead of using a webapi.add_torrents method, it uses a core.add_torrent_magnet or core.add_torrent_file method.

After going over my deluge logs I found that when I try and add a torrent I get the following error in the logs:

Unable to decode torrent file!: unexpected end of file in bencoded string

As you suggested that it could be expecting the magnet:?xt=urn:btih: denotion, I thought I would try an alternative method of adding the torrent. This time I tried

1. Downloading the torrent file
2. Importing the base64 library in to IDLE and then running the file through the base64encode method in the repo you referenced.

Step 2 gave me the following string QzpcVXNlcnNcdGd3dzFcRGVza3RvcFxQb2tlbW9uXCBMZXRzXCBHb1wgRWV2ZWVcIFVwZGF0ZVwgndjFcIDBcIDFcIE5TVy1WRU5PTS50b3JyZW50n, I then used the build_request_payload function to build the JSON with this data instead which gave me '{"id": 1, "method": "webapi.add_torrent", "params": ["QzpcVXNlcnNcdGd3dzFcRGVza3RvcFxQb2tlbW9uXCBMZXRzXCBHb1wgRWV2ZWVcIFVwZGF0ZVwg\ndjFcIDBcIDFcIE5TVy1WRU5PTS50b3JyZW50\n", {"download_location": "/home1/riiskyyy/downloads/"}]}'.

However even after this, the torrent was still not added to deluge, I tried using the same core.add_torrent_magnet & core.add_torrent_file methods (the methods that sonarr uses), with the link and the decoded file respectivley, but still the same result no error but nothing added to deluge.

When I looked back over my deluge logs, the times I had tried to add the file using the decoded utf-8 output of the torrent file I had different errors:

There was an error decoding the filedump string!
Incorrect padding
Traceback (most recent call last):
  File "/opt/deluge-1.3.15/local/lib/python2.7/site-packages/deluge-1.3.15-py2.7.egg/deluge/core/core.py", line 249, in add_torrent_file
    filedump = base64.decodestring(filedump)
  File "/usr/lib/python2.7/base64.py", line 328, in decodestring
    return binascii.a2b_base64(s)
Error: Incorrect padding

Once I removed the n from the decoded string it just went back to the Unable to decode torrent file!: unexpected end of file in bencoded string error.

I have also tired without adding a path for where to download the torrent to so it would just use the default set in deluge. But this also did not work.

I’m not sure what the issue is here as the following are all true:

1. Sonarr can add torrents to deluge using the Jackett download link, but I cannot using JSON.
2. I can add the torrent using the same URL manually through the web ui.
3. If I manually download the file and decode it with the base64decode function it does not work.
4. The deluge core.add_torrent_magnet and core.add_torrent_file method produce the same results as 1 & 3.