webtorrent: file.deselect supposedly not working

Heard from someone on IRC that after calling file.deselect on a file, it still gets fully downloaded.

About this issue

  • Original URL
  • State: closed
  • Created 12 years ago
  • Comments: 23 (6 by maintainers)

Most upvoted comments

I can confirm in v0.108 that torrent.deselect(0, torrent.pieces.length - 1, false) doesn’t work. And I remember this line working time ago. Is there any alternative to this? any workaround?

Yeah, the deselect() API is janky.

You have to deselect the whole torrent first, then select individual files.

Here’s how we made it work in WebTorrent Desktop:

  // Remove default selection (whole torrent)
  torrent.deselect(0, torrent.pieces.length - 1, false)

  // Add selections (individual files)
  for (let i = 0; i < selections.length; i++) {
    const file = torrent.files[i]
    if (selections[i]) {
      file.select()
    } else {
      console.log('deselecting file ' + i + ' of torrent ' + torrent.name)
      file.deselect()
    }
  }

@feross we should probably improve this API before WebTorrent 1.0, because afterward we don’t want to change it.

Is there a reason to let people deselect individual blocks within a file? What if the API only allowed selecting which files in a torrent to download?

Here’s an API proposal

Remove the select and deselect methods from File and Torrent.

Let users specify which files they want when adding a torrent:

client.add('magnet://...', { fileSelections: [true, true, false, true] })

When you do that, the files that aren’t selected are never created on disk.

Let users specify which files they want after adding a torrent:

torrent.setFileSelections([true, true, false, true])

Doing that doesn’t delete any files. Partially downloaded files just stay there, unless the user deletes them separately. This lets you pause and resume individual files in a torrent.

This alternate API would require the library to do a bit more work to keep track of pieces overlapping a file boundary. It’s simpler, though. You always declare what you want to download for the whole torrent, instead of calling select() and deselect() on individual files and piece ranges. This API also lets you download a torrent without creating all the files on disk.

@feross

Fixed by this:


const file = torrent.files[fileIndex];

// Deselect all files on initial download
torrent.files.forEach(file => file.deselect());
torrent.deselect(0, torrent.pieces.length - 1, false);

// Select file with provided index
if (file) torrent.select(file._startPiece, file._endPiece, false);

Doing tests and tests and tests, I can confirm this seems to work:

        let content_torrent = content_wt.add(content_magnet_link, function(this_torrent)
	{
		//
	    // Criteria to select files, first we deselect all of them
	    //
	    this_torrent.files.forEach(function (file, index, arr) 
	    {
	        file.deselect();
	    });
	    /*
	    // Will help to stop the download as I'm still waiting for the finished implementation
	    this_torrent.deselect(0, this_torrent.pieces.length - 1, false);  
	    */
	});

    content_torrent.on('ready', function() 
    {
    	//
		// DEBUG
		//
		function on_progress() 
	    {

	        console.log(`d: ${pretty_bytes(content_torrent.downloadSpeed)}/s - u: ${pretty_bytes(content_torrent.uploadSpeed)}/s - u: `);
	    }

    	on_progress();
    	setInterval(on_progress, 500);
	}

I started implementing @dcposch proposal, in the fashion described by @kocoten1992, with a list of wanted files. It’s over there - it needs a bit more testing but seem to work fine on my side.

Just to give my two cents your comment above @detarkende, I would say A and B are both valid depending on the scenario.

Given a torrent which has the piece layout:

Piece 1 | <---file 1---> <---file 2---> |
Piece 2 | <--------- file 3 ----------> |

Calling file1.select() would select piece 1. Calling file2.select() would select piece 1.

Calling file1.deselect() would currently lead to file2 being deselected which is not desired behaviour as both files depend fully on the same piece.

Say we had the further piece layout:

Piece 1 | <--------- file 1 ----------> |
Piece 2 | <---file 1---> <---file 2---> |
Piece 3 | <--------- file 2 ----------> |

Calling file1.select() would select piece 1 and 2. Calling file2.select() would select piece 2 and 3.

Both file1 and file2 depend on piece 2, but deselecting either would currently cause a piece required by another file to be deselected, this again would not be desired behaviour.

I believe Alex’s changes are an attempt to allow overlapping ranges and priorities, so that multiple selections can include the same piece, and the algorithm used will flatten this to the linear ranges based on ranges and priorities required, e.g.

In the second example above, calling file1.select() and torrent.select(2, 3, 1) would look like this, as the higher priority takes precedent:

[{ from:1, to: 1, priority: 0 }, { from: 2, to: 3, priority: 1 }]

But calling file1.select() and file2.select() could look like either:

[{ from:1, to: 1, priority: 0 }, { from: 2, to: 3, priority: 0 }]
// or
[{ from:1, to: 2, priority: 0 }, { from: 3, to: 3, priority: 0 }] 
// Note: It's probably easier to continue from the existing selection as to not have to update + insert.

And when deselecting content, all individual ranges are evaluated, so calling file1.deselect() would update the selections to look like:

// With priorities
[{ from:1, to: 1, priority: 0 }, { from: 2, to: 3, priority: 1 }]
// to
[{ from: 2, to: 3, priority: 1 }]

// Without priorities
[{ from:1, to: 2, priority: 0 }, { from: 3, to: 3, priority: 0 }] 
// to
[{ from: 2, to: 3, priority: 0 }] 

and calling file2.deselect() would leave look like:

// With priorities
[{ from:1, to: 2, priority: 0 }, { from: 3, to: 3, priority: 1 }]
// to
[{ from:1, to: 2, priority: 0 }, { from: 3, to: 3, priority: 1 }] // Same

// Without priorities
[{ from:1, to: 2, priority: 0 }, { from: 3, to: 3, priority: 0 }] 
// to
[{ from:1, to: 2, priority: 0 }] 

Note: The lack of changes in // With priorities is due to file.deselect() only deselecting with a priority of 0, not 1, which typically means it’s being used by a stream that we don’t want to interrupt.

If that all makes sense?

@RangerMauve Unfortunately you aren’t going to be able to completely prevent the files themselves from being created. The real problem is that Webtorrent actually has no concept of selecting a file. Instead, you are only allowed to select the pieces of the torrent which you wish to download. Since a piece can overlap multiple files (a piece may contain the end of one file and the beginning of the next), Webtorrent will end up writing the bytes to their corresponding files, which will likely result in creating files that are directly adjacent to the selected files.

So the result is that you may get some extra files created that you didn’t want, but the size of them should be fairly negligible as they shouldn’t really be more than a few kb in size (256kb in the worst case if I’m correct, but I could be wrong). Your best bet is to maintain the selected files yourself (you’ll have to anyway to figure out which are selected), and just throw the other files away when you’re done with them. If you plan on seeding the torrent, you’ll want to keep them around as they have parts of the pieces.

I also want to note that I had found issues with using so = -1 due to how it was searching for selected torrents. I recommend using a non-numerical value (something like so = '-')

Thanks @PavelShar I just tried that with a full example https://codepen.io/Jolg42/pen/qBOeYej?editors=1010

const client = new WebTorrent()

const torrentId = 'https://webtorrent.io/torrents/wired-cd.torrent'

client.add(torrentId, function (torrent) {
  torrent.on('done', function(){
    console.log('torrent finished downloading')
  })

  torrent.on('download', function (bytes) {
    console.log('total downloaded: ' + torrent.downloaded)
    console.log('progress: ' + torrent.progress)
  })

  console.log('Torrent name:', torrent.name)
  console.log('Files:')
  torrent.files.forEach(file => {
    console.log('- ' + file.name)
  })
  
  // Deselect all files on initial download
  torrent.files.forEach(file => file.deselect());
  torrent.deselect(0, torrent.pieces.length - 1, false);

  // Torrents can contain many files. Let's use the .mp4 file
  const file = torrent.files.find(function (file) {
    console.log(`We will only download and play ${file.name}`)
    file.select()
    return file.name.endsWith('.mp3')
  })

  // Display the file by adding it to the DOM. Supports video, audio, image, etc. files
  file.appendTo('body', {autoplay: true, muted: true})
})