Something behind yield

Today I spent quite some time on digging into C# yield keyword. It is one of the hard areas in C# for me to understand. I started by reading the C# reference but it didn’t help me too much, until I found this article on MSDN magazine.

 

 

Overview

If you need to return a collection in one function, you can choose to return a collection (such as List<T>) directly, or return an IEnumerator for the outside loop to iterate. The first one is easy to implement but there might have performance defects when the collection is huge – for every call of our function there will be a List<T> object generated on GC heap, even if you only want to process the first several objects. The 2^nd solution is much simpler and more memory effective however there are several issues attached with it:

1)    Hard to implement IEnumerators for complicated data structures like binary trees. Our IEnumerator implementation has to maintain the state of the current visit and the code is really complex and error leading in this case.

2)    If you are going to provide several different iterators for a collection, our code will be bloated with various iterator implementations.

 

yield keyword (introduced in C# of .Net 2.0) is provided to address the above two issues. With yield there is no need for you to worry about remembering state for the iterator anymore. All you need to do is to tell what to yield and the compiler will automatically generate the IEnumerator implementation for you automatically.

 

public class EnumerableTest: IEnumerable<int>

{

IEnumerator<int> IEnumerable<int>.GetEnumerator()

{

yield return 0;

yield return 1;

for(int i = 2; i < 5; i++)

{

yield return i;

if (i > 3)

{

yield break;

}

}

}

IEnumerator IEnumerable.GetEnumerator()

{

return ((IEnumerable<int>)this)GetEnumerator();

}

}

 

A simple way to understand yield is as follows: Compiler generated code returns each value you yield in your code (with regards to the order they appear in the code), and pause the execution of the method from that place. When next iteration comes, the method will resume execution from where it is paused and next value is returned.

 

Iterator Implementation

The C# compiler will generate a nested class for the code containing the yield keyword. The class is an implementation of the IEnumerator<T> interface and encapsulates the state needed to iterate within the parent class – here a simple state machine is used to achieve this. When the iterator is first called a new iterator object is created and it’ll be used throughout the remaining iterations. The iterator object doesn’t persist across loops so it is safe for you to call foreach again because you’ll get a brand new iterator object to start the new iteration.

 

There are plenty of articles about this so I don’t want to put more details here.

 

Limitations Explained

yield can appear in any iterator block, where the iterator block can be function, property, indexer etc. There are some limitations where you can use yield though:

1)    You cannot have a return in the function or property where yield is used.

This is obvious because a return will break the iteration improperly.

2)    yield cannot be used in a try block which has a corresponding catch block (finally block is OK).

This is not very intuitive but not very hard to understand. It’s all about the complexity of supporting this.

IEnumerable<int> Content()

{

      try

{

      int val = CalculateVal(…);

      yield return val;       // You’ll get a compilation error here.

}

catch (SomeException e)

{

…

}

}

Suppose the CalculateVal throws an exception, there will be nothing to return for this iteration, which breaks the iterator contract. A possible solution is to move to calculate the next value. Remember this needs to be tracked by the state machine – you can guess out the complexity I believe. Furthermore, is this solution really the wanted behavior for everybody? I doubt.

3)    yield cannot be used within anonymous methods.

Hmm, with this limitation we lose the ability to use yield in lambda expressions. What a pity!

foreach (int sth in (x => { for (int y=0; y<4; y++) { yield return 2*y; y++; } }))  // You’ll get another 2 compilation errors here.

{

      …

}

It is said that Microsoft is going to support using yield within anonymous methods in the future. I am looking forward to that.

 

Return Type of Iterator Block

You can return either IEnumerator or IEnumerable for your iterator block using yield. The hidden state machine is always an IEnumerator, and it’s exactly the same code whether you specify IEnumerator or IEnumerable.

Most of the time specifying IEnumeralbe is better because the compiler will automatically generate some kind of “outer” class and provides an implementation of GetEnumerator, which simplifies your code a lot.

 

When would you want to return IEnumerator then? The answer is: it’ll be very useful if you would like to return multiple iteration methods. For example you want to provide pre-order, in-order and post-order iteration for a binary tree, you might use following code to achieve this. This thread has more about this.

 

public class BinaryTree

{

public IEnumerator<Node> PreOrder()

{

…yield return…

}

 

public IEnumerator<Node> PreOrder()

{

…yield return…

}

 

public IEnumerator<Node> PreOrder()

{

…yield return…

}

}