The “For in” style of enumeration in Delphi was introduced in Delphi 2005. The idea is to make enumeration over a collection of things easier and cleaner by removing the need for the iterator variable, removing the code for indexing and removing the code for boundary checking. If you think you know everything about enumeration, hold on to your hat. You may be in for a pleasant surprise.

We will start with the basics and then progress to custom enumerators. Seasoned “enumerophiles” may prefer to skip to the “Custom Enumeration over Objects” section. Let’s look at a simple example. Say we’ve got a list of strings, and we want to show them to the user. Here is how we would do it before and after this feature.

Enumeration by index

procedure EnumDemo1;
var
  SomeStrings: array of string;
  s: string;
  i: integer;
begin
  Populate( SomeStrings);
  for i := 0 to Length( SomeStrings) - 1 do
  begin
    s := SomeStrings[i];
    Dialogs.ShowMessage( s)
  end
end;

Enumeration by for-in

procedure EnumDemo2;
var
  SomeStrings: array of string;
  s: string;
begin
  Populate( SomeStrings);
  for s in SomeStrings do
    Dialogs.ShowMessage( s);
end;

Looking at the details of the above, you see that for-in enumeration is not exactly earth-shattering. We have dropped a variable declaration, the for-loop is a little cleaner and indexing is compiler-magiced away. But you’ve got the wrong idea if you think that the key selling point of for-in enumeration is these simple syntactic abbreviations. The real selling point is greater than that.

Stand back from the details of the code, and think about what we are doing at an abstract level. We are looping through a collection of strings and doing something with the members. From this perspective, the mechanism of indexing is a pesky irrelevant detail. What manner of collection it is (array, list, something-else) is a pesky irrelevant detail. Boundary checking? – another irrelevant artefact of implementation. What we have achieved in EnumDemo2() over EnumDemo1(), is the holy grail of language design – a closer alignment of semantics with syntax.

Standard Enumerables

So what can we enumerate over, out-of-the-box? The following VCL types are for-in enumerable? Member types are as you would expect.

Fundamental and compound types – Enumerables

• Dynamic arrays
• Static arrays
• string/ ansistring/ utf8string
• sets

Sets are a special case. Enumeration only visits members that are in the set, as opposed to the base range.

Object Enumerables

• TArray<T>
• TStrings
• TList
• TList<T>
• TObjectList
• TObjectList<T>
• TCollection
• TQueue<T>
• TStack<T>
• TDictionary<T>
• TInterfaceList
• TObjectStack<T>
• TComponent
• TFields
• TActionList
• TTreeNodes
• TListItems
• TToolBar
• TMenuItem
• A large suite of RTTI properties.

Interface pointer Enumerables

• IInterfaceListEx

What’s missing?

So what is missing from Delphi out-of-the-box for-in enumeration support?

Perhaps in a future version of Delphi we will get enumeration support for:

• Enumeration types. With some custom declarations, enumeration of “enumerated types” has been solved by Jeroen W. Pluimers
• Set inversion. Currently, there is no way to for-in enumerate over members NOT in set. Wouldn’t it be handy if we had a generic function to invert sets.
• Character Streams. Ever wanted to parse a text file character by character? Wouldn’t it be nice if we could use for-in?
• Reverse order. Have you ever wanted to traverse in reverse order? How useful would this be?
• TBits
• TDataset – How many times have you written a while-not-eof loop? For us older developers, the number is probably astronomical.
• Predicates. Ever wanted to traverse a collection, but only for those members that met some specified condition. Sure we can do this with a simple if statement, but this pattern is so common, the thought of an XSLT style syntax mechanism for predicates is enticing. Imagine if we could do something like FantasyEnum4()

Predicated enumeration as it is now…

procedure EnumDemo3( Collection: TObjectList<TSomeClass>);
var
  Member: TSomeClass;
begin
  for Member in Collection do
    if SomeCondition( Member) then
      Member.SomeAction
end;

Fantasy enumeration supporting predicates

procedure FantasyEnum4( Collection: TObjectList<TSomeClass>);
var
  Member: TSomeClass;
begin
  for Member in Collection ! SomeCondition( Member) do
    Member.SomeAction
end;

Generics and Enumeration

Generic classes and for-in enumeration are a good match-up. Take a look at the syntactic synergy in EnumDemo3 above. If we wanted to do the same in Delphi 7, we would probably write something like EnumDemo5() below…

procedure EnumDemo5( Collection: TObjectList);
var
  i: integer;
  Member: TSomeClass;
begin
  for i := 0 to Collection.Count - 1 do
  begin
    Member := Collection[i] as TSomeClass;
    if SomeCondition( Member) then
      Member.SomeAction
  end
end;

Custom Enumeration over objects

So much for the basics. Now here is where it gets really interesting. With some customisation you can for-in enumerate over any object of any class.

You can enumerate over objects, records and interface pointers. The enumerators also can either be objects, records or interface pointers. And the collection members of the enumeration can by any type including fundamentals, compounds, objects, interface pointers, anonymous functions, even class references and type references.

When the compiler sees a code like

for Member in Collection do

.. and Collection is an object, it compiles by the following algorithm..

1. Look for a public function of name GetEnumerator(). The function return type must be a class, record or interface pointer type. Protected and below methods do not count. Class functions are also discounted.
2. If class, Examine the declaration of the class. There must be public non-class function MoveNext(), with signature…

   function MoveNext: boolean;
   

3. Also in this class, there must be a public property named Current. The read accessor method for Current may be any thing, even strict private.
4. The type for the current property must be equal to the declared type of the enumerated value (‘Member” in our above example), or, if the type is an object type, the current class type may be a descendant of declared member class. If the member type is an interface type, then the current property interface type may be an interface descendant of the declared member interface type. It is not required for interface types to have syntactically bound guids.
5. The MoveNext() and Current are utilised as follows…

With declarations …

TMyEnumerator = object
  strict private
    function GetCurrent: TMemberClass;
  public
    property Current: TMemberClass read GetCurrent;
    function MoveNext: Boolean;
  end;

TMyCollection = object
  public
    function GetEnumerator: TMyEnumerator;
  end;

…this procedure …

procedure EnumDemo6a( Collection: TMyCollection);
var
  Member: TMemberClass;
begin
  for Member in Collection do
    SomeAction( Member)
end

… is compiled as if we wrote instead…

procedure EnumDemo6b( Collection: TMyCollection);
var
  Member: TMemberClass;
  Enumerator: TMyEnumerator;
begin
  Enumerator := Collection.GetEnumerator;
  try
    while Enumerator.Next do
    begin
      Member := Enumerator.Current;
      SomeAction( Member)
    end
  finally
    Enumerator.Free
  end
end

If the Enumerator is an interface pointer, then instead of being freed, it is released (reference count decremented). If the Enumerator is a record, then instead of being freed, it is record finalized and de-scoped.

If you are making custom enumerators, you may find it convenient to leverage these base classes and interface pointer types from the VCL…

  IEnumerator = interface(IInterface)
    function GetCurrent: TObject;
    function MoveNext: Boolean;
    procedure Reset;
    property Current: TObject read GetCurrent;
  end;

  IEnumerable = interface(IInterface)
    function GetEnumerator: IEnumerator;
  end;

  IEnumerator = interface(IEnumerator)
    function GetCurrent: T;
    property Current: T read GetCurrent;
  end;

  IEnumerable = interface(IEnumerable)
    function GetEnumerator: IEnumerator;
  end;

  TEnumerator = class abstract
  protected
    function DoGetCurrent: T; virtual; abstract;
    function DoMoveNext: Boolean; virtual; abstract;
  public
    property Current: T read DoGetCurrent;
    function MoveNext: Boolean;
  end;

  TEnumerable = class abstract
  protected
    function DoGetEnumerator: TEnumerator; virtual; abstract;
  public
    function GetEnumerator: TEnumerator;
  end;

Custom Enumeration over records and interface pointers.

You can enumerate over records and interface pointers too. It works the same way as classes, except that the restriction that the GetEnumerator method be public is not applicable.

Custom Enumeration with helpers

There is a natural synergy between class helpers and custom enumeration. See the next section for examples.

Some incredibly useful examples

Parse a character stream, line by line

With a class helper and custom enumeration, we can conveniently pass a text stream, line by line (or character by character just as easily), like so…

procedure ParseLines( TextStream: TStream);
var
  Line: string;
begin
  for Line in TestStream do
    ParseLine( Line)
end;

One possible implementation to achieve this (with a UTF-8 encoded stream) might be….

TTextStreamHelper = class helper for TStream
  public
    function GetEnumerator: TEnumerator;
  end;

function TTextStreamHelper.GetEnumerator: TEnumerator;
begin
  result := TLineReader.Create( self);
end;

TLineReader = class( TEnumerator)
  private
    coTextStream: TStream;
    ciPosition: Int64;
    csCurrent: string;

  protected
    function DoGetCurrent: string; override;
    function DoMoveNext: Boolean; override;

  public
    constructor Create( poTextStream: TStream);
  end;

constructor TLineReader.Create( poTextStream: TStream);
begin
  coTextStream := poTextStream;
  ciPosition   := coTextStream.Position;
  csCurrent    := ''
end;

function TLineReader.DoGetCurrent: string;
begin
  result := csCurrent
end;

function TLineReader.DoMoveNext: Boolean;
var
  Ch, Ch2: Ansichar;
  sWorking: UTF8String;
  L: integer;
begin
  sWorking := '';
  result   := False;
  while coTextStream.Read( Ch, 1) = 1 do
  begin
    result := True;
    if (Ch <> #13) and (Ch <> #10) then
    begin
      L := Length( sWorking) + 1;
      SetLength( sWorking, L);
      sWorking[L] := Ch;   // Adds a code-unit NOT a character!
    end
    else
    begin
      if (coTextStream.Read( Ch2, 1) = 1) and
           (((Ch = #13) and (Ch2 <> #10)) or
            ((Ch = #10) and (Ch2 <> #13))) then
        coTextStream.Seek( -1, soCurrent);
      break
    end;
  end;
  csCurrent := UTF8ToString( sWorking)
end;

XML Navigation

See my previous blog entry Dances with XML

Data set

I haven’t yet developed a class helper for traversing a TDataSet, but it strikes me as something that would be incredibly convenient. I invite the reader to have a go. Tell us about your efforts in the comment stream.

Third party support

Mike Lischke’s TVirtualTree, (a tree and list view component), is probably the worlds’s most popular third party component, and for good reason. TVirtualTree supports for-in enumeration.

I suspect that most readers of this blog are also readers of Nick Hodge’s “A man’s got to know his limitations“ blog. In this entry, Nick introduces the Delphi Spring Framework’s extention to IEnumerable. The entry is worth reading, including Nick’s answer to the originating question on StackOverflow.

Restrictions

• One would expect that the member variable would be syntactically required to be a local variable. Surprisingly this is not the case. However, I advise against non-local member variables, as this is contrary to the semantics.
• One would expect that member variables should not be capturable by closures. And this too is not the case. Again, don’t do it. The outcome would be just too messy.
• Within the loop, the member variable cannot be assigned to.
• The member variable can be referenced, both before and after the loop. Unlike a for-to loop variable, it’s value is indeed defined after loop exit, provided that the loop has passed at least one iteration.