svelte: Allow `#each` block to handle arbitrary non array like iterables

Doesn’t that defeat the performance benefit of Set and Map, though, when dealing with massive data sets?

In native JS, we can iterate over sets and maps using for…of and retain those perf benefits without needing to port their contents to an array. It would be ideal to keep those benefits.

const myMap = new Map();
myMap.set('key1', 'value1');
myMap.set('key2', 'value2');
myMap.set('key3', 'value3');

I ran some benchmarks just to demonstrate this difference, and Map iteration is faster across all (by over 50%):

	JSBench.me results	Perflink results	JSBEN.CH results
for..of(Map)	fastest ✅	fastest ✅	fastest ✅
for..of(Array.from(Map))	62.03% slower 🔻	~60% slower 🔻	62.06% slower 🔻
for..of(Array.from(Map.entries()))	63.42% slower 🔻🔻	~62% slower 🔻🔻	63.63% slower 🔻🔻
	link to test	link to test	link to test

The fact alone that taking advantage of the perf benefits is so attainable and not something other frameworks have done yet is a huge opportunity for Svelte, and I think we’d be falling short to not take a key stake in big (and attainable) performance wins like this.

I think for..of loops require an iterable anyway, so by swapping to a for..of loop here, you’d naturally be supporting both Maps and Sets and other iterable data structures.

Implemented in https://github.com/sveltejs/svelte/pull/8626

Would love to see this added to the roadmap for v4

i cant overstate how exciting this is. this is the one problem that forces me to patch svelte at the moment. basically i have an arraylike js proxy object that unfortunately returns typeof as “function” for unrelated reasons

at the moment the each statement in svelte checks if something is an object which is not true and its also not clear why only objects with a length attribute should be allowed, instead arg?.length seems like reasonable check for array like.

currently the check is

arg && typeof arg === 'object' && 'length' in arg

additionally supporting generic iterables would solve a big pain for me as i often iterate of entries where the number is not known before the iteration ends, this forces me to guess the length and then run into all kinds of issues when when its off

@joeally Exactly. The Map and Set are not special cases. Every iterable can be iterated over using the same for..of loop style. The only exception is the “array-like” case, which is easy enough to support performantly, and one additional but optional exception is the plain array type.

Array types are iterable in the same way that Map, Set, and generators are, but they actually iterate more performantly if iterated over using a plain for loop, so I use a !Array.isArray(obj) check to push array types into the second block alongside the array-like type.

We can loop over someMap.entries() and someSet.values() alternatively, but generally speaking, it’s actually more performant to just loop over maps and sets directly without using entries() and values(), which is also simpler syntax, so win-win. 🎉

Not sure why I’m not seeing @eabald’s comment here anymore, so reposting it here. As it echos the same comments many of us have voiced before:

I use maps exclusively in the stores for record collections because it’s much faster to get and update an entry by ID than it would be to retrieve find the index in the array - so I’m also ending up iterating maps with each.

I’d put my vote that Maps deserve dedicated processing while converting all other iterables to arrays.

I think maps and sets both really need this. This is a massive performance deficit for Svelte and can easily be avoided as demonstrated earlier and still support an index, so I’d also argue that the lack of a natural iterable index is a weak argument as well.

Also, for many iterables, like maps and sets, you can use size instead of length and get the expected results.

The helper function I outlined above is a single function that accounts for this and manages all cases. With a small tweak, you could limit the performance boost to maps and sets for their size property and convert everything else to an array.

Were there any other downsides to using that?

As @lucidNTR mentioned, users can also do that conversion to arrays explicitly if it’s what they want, but in most cases, I explicitly work with maps and sets because they’re generally more performant with large datasets. The current implementation flips that upside down and is slower than using an array to begin with.

@dummdidumm by that argument you could just say people should wrap iterables with array.from in application layer, that way there would be 0 lines of code change in svelte and we would not break expectations. supporting iterables means by definition allowing to start rendering pages of entries from the iterable without accessing length, if this is too much work at the moment this should just be postponed but this will lead to many issues

@dummdidumm Thanks for all your work on this!

Did you see my earlier comments about iterating over iterable values directly (maps, sets, generators, etc.), which greatly improves performance?

If I’m reading the merged PR accurately, it looks like those would all now be converted to arrays using Array.from, which is significantly slower.

• https://github.com/sveltejs/svelte/issues/7425#issuecomment-1461121464
• https://github.com/sveltejs/svelte/issues/7425#issuecomment-1462509681
• https://github.com/sveltejs/svelte/issues/7425#issuecomment-1468603617

Amazing to see this change in Svelte 4.0. Thanks for all of the hard work @dummdidumm and anyone else who worked on it.

@brandonmcconnell - I’d prefer the user to have to call .entries(). IMO the {#each} should work with any iterable (see: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols).

{#each} should not have a special case for Map and Set.

Yes you’re right. I didn’t know that Map itself can be iterated over.

I agree that a solution like 'length' in foo ? foo : Array.from(foo) would be ideal, though I think we could still preserve the performance benefits of Maps and Sets with minimal effort.

What “ability to do minimal DOM updates” would be lost by switching to use for..of loops?

If we use a lightweight abstraction like the iterateOver function below, we can keep all the benefits you mentioned and also support all iterables (e.g. maps, sets, generators, arrays, etc.) and array-likes.

Implementation

— see on TS Playground

function isIterable<T>(obj: Iterable<T> | ArrayLike<T>): obj is Iterable<T> {
  return typeof (obj as Iterable<T>)[Symbol.iterator] === 'function';
}

function iterateOver<T>(obj: Iterable<T> | ArrayLike<T>, expose: Function) {
  if (isIterable(obj) && !Array.isArray(obj)) {
    /* non-array iterables (array iteration using a traditional `for` loop is more performant) */
    let i = 0;
    for (const elem of obj) {
      expose(elem, i++);
    }
  } else {
    /* arrays and array-likes */
    const indexedObj: ArrayLike<T> = (
      'length' in obj && typeof obj.length === 'number'
        ? (obj as { length: number })
        : Array.from(obj)
    );
    for (let i = 0; i < indexedObj.length ?? 0; i += 1) {
      expose(indexedObj[i], i);
    }
  }
}

In ☝🏼 this example, I am using an imaginary “expose” function, but that would just be replaced by whatever logic or helper function you use to slot in the HTML for each {#each} block iteration.

Tests

— see all tests on TS Playground (same link as above)

Here are some tests for that function, each of which works as expected:

• Array-like example { length: 3 }

  const arrayLikeExample = { length: 3 };
  
  iterateOver(arrayLikeExample, (_: undefined, i: number) => console.log(_, i));
    // -> undefined 0
    // -> undefined 1
    // -> undefined 2
  

• Array example ['value1', 'value2', 'value3']

  const arrayExample = ['value1', 'value2', 'value3'];
  
  iterateOver(arrayExample, (val: string, i: number) => console.log(val, i));
    // -> 'value1' 0
    // -> 'value2' 1
    // -> 'value3' 2
  

• Set example {'value1', 'value2', 'value3'}

  const setExample = new Set();
  setExample.add('value1');
  setExample.add('value2');
  setExample.add('value3');
  
  iterateOver(setExample, (val: string, i: number) => console.log(val, i));
    // -> 'value1' 0
    // -> 'value2' 1
    // -> 'value3' 2
  

• Map example {'key1' => 'value1', 'key2' => 'value2', 'key3' => 'value3'}

  const mapExample = new Map();
  mapExample.set('key1', 'value1');
  mapExample.set('key2', 'value2');
  mapExample.set('key3', 'value3');
  
  iterateOver(mapExample, ([key, val]: [string, string], i: number) => console.log(key, val, i));
    // -> 'key1' 'value1' 0
    // -> 'key2' 'value2' 1
    // -> 'key3' 'value3' 2
  

• Generator example 'value1' => 'value2' => 'value3'

  function* generatorExample(): IterableIterator<string> {
    let current = 1;
    while (current <= 3) {
      yield `value${current}`;
      current++;
    }
  }
  
  iterateOver(generatorExample(), (val: string, i: number) => console.log(val, i));
    // -> 'value1' 0
    // -> 'value2' 1
    // -> 'value3' 2
  

The DOM is slow, and I am almost certain that losing the ability to do minimal DOM updates would far outweigh what we’d gain by only iterating through the array once with for of. Using for of internally would also mean that we could no longer directly support iterating over the array-like object { length: 42 }, which is something we’ve encouraged people to do in the past when all they really care about is the number of times something appears in the DOM. We would probably want to have something like 'length' in foo ? foo : Array.from(foo) so we don’t lose performance for things that already are arrays or other array-likes.

Rich Harris has recently talked about supporting this too: https://youtu.be/dB_YjuAMH3o?t=4m18s