Variable iterating on itself - different behavior with different types

Please see edits below!

For Each edits also added below under Edit2

More edits about ForEach and Collections at Edit3

One last edit about ForEach and Collections at Edit4

A final note about iteration behavior at Edit5

Part of the subtlety of this odd behavior in the semantics of variant evaluation when used as a loop control variable or terminating condition.

In a nutshell, when a variant is the terminating value, or the control variable, the terminating value is naturally re-evaluated by the runtime with each iteration. A value type, however, such as an Integer, is pushed directly, and thus not re-evaluated (and its value doesn't change). If the control variable is an Integer, but the terminating value is a Variant, the Variant is coerced to an Integer on the first iteration, and pushed similarly. The same situation arises when the terminating condition is an expression involving a Variant and an Integer - it's coerced to an Integer.

In this example:

Dim v as Variantv=4for v= 1 to v  Debug.print v,next

The variant v is assigned an integer value of 1, and the loop termination condition is re-evaluated because terminating variable is a variant - the runtime recognizes the presence of the Variant reference and forces re-evaluation with each iteration. As a result, the loop completes because of the in-loop reassignment. Because the variant now has a value of 1, the loop termination condition is satisfied.

Consider this next example:

Dim v as variantv=4for v=1 to v-0   Debug.Print v,next 

When the terminating condition is an expression, such as "v - 0", the expression is evaluated and coerced to a regular integer, not a variant, and thus its hard value is pushed to the stack at runtime. As a result, the value is not re-evaluated upon each loop iteration.

The other interesting example:

Dim i as IntegerDim v as variantv=4For i = 1 to v   v=i-1   Debug.print i,next

behaves as it does because the control variable is an Integer, and thus the terminating variable is coerced to an integer as well, then pushed to the stack for iteration.

I cannot swear these are the semantics, but I believe the terminating condition or value is simply pushed onto a stack, thus the integer value is pushed, or the Variant's object reference is pushed, thus triggering the re-evaluation when the compiler realizes a variant holds the terminating value. When the variant gets reassigned within the loop, and the value is re-queried as the loop completes, the new value is returned, and the loop terminates.

Sorry if that's a little muddy, but it's kinda late, but I saw this and couldn't help but take a shot at an answer. Hope it makes some sense. Ah, good ol' VBA :)

EDIT:

Found some actual info from the VBA language spec at MS:

The expressions [start-value], [end-value], and [step-increment] are evaluated once, in order, and prior to any of the following computations. If the value of [start-value], [end-value], and [step-increment] are not Let-coercible to Double, error 13 (Type mismatch) is raised immediately. Otherwise, proceed with the following algorithm using the original, uncoerced values.

Execution of the [for-statement] proceeds according to the following algorithm:

1. If the data value of [step-increment] is zero or a positive number, and the value of [bound-variable-expression] is greater than the value of [end-value], then execution of the [forstatement] immediately completes; otherwise, advance to Step 2.

2. If the data value of [step-increment] is a negative number, and the value of [bound-variable-expression] is less than the value of [end-value], execution of the [for-statement] immediately completes; otherwise, advance to Step 3.

3. The [statement-block] is executed. If a [nested-for-statement] is present, it is then executed. Finally, the value of [bound-variable-expression] is added to the value of [step-increment] and Let-assigned back to [bound-variable-expression]. Execution then repeats at step 1.

What I gather from this is that the intent is for the terminating condition value to be evaluated once and once only. If we see evidence that changing that value changes the behavior of the loop from its initial condition, it is almost certainly due to what might be termed informally as accidental re-evaluation because it's a variant. If it's unintentional, we can probably only use anecodtal evidence to predict its behavior.

If as the runtime evaluates a loop's start/end/step values, and pushes the "value" of those expressions onto the stack, a Variant value throws a "byref wrench" into the process. If the runtime does not first recognize the variant, evaluate it, and push that value as the terminating condition, curious behavior (as you are showing) would almost certainly ensue. Exactly how VBA handles variants in this case would be a great task for pcode analysis, as others have suggested.

EDIT2: FOREACH

The VBA spec again provides insight into the evaluation of ForEach loops over collections and arrays:

The expression [collection] is evaluated once prior to any of the >following computations.

1. If the data value of [collection] is an array:

   If the array has no elements, then execution of the [for-each-statement] immediately completes.

   If the declared type of the array is Object, then the [bound-variable-expression] is Set-assigned to the first element in the >array. Otherwise, the [bound-variable-expression] is Let-assigned to the >first element in the array.

   After [bound-variable-expression] has been set, the [statement-block] >is executed. If a [nested-for-statement] is present, it is then executed.

   Once the [statement-block] and, if present, the [nested-for-statement] >have completed execution, [bound-variable-expression] is Let-assigned to >the next element in the array (or Set-assigned if it is an array of >Object). If and only if there are no more elements in the array, then >execution of the [for-each-statement] immediately completes. Otherwise, >[statement-block] is executed again, followed by [nested-forstatement] if >present, and this step is repeated.

   When the [for-each-statement] has finished executing, the value of >[bound-variable-expression] is the data value of the last element of the >array.

2. If the data value of [collection] is not an array:

   The data value of [collection] must be an object-reference to an >external object that supports an implementation-defined enumeration >interface. The [bound-variable-expression] is either Let-assigned or >Set-assigned to the first element in [collection] in an >implementation->defined manner.

   After [bound-variable-expression] has been set, the [statement-block] >is executed. If a [nested-for-statement] is present, it is then executed.

   Once the [statement-block] and, if present, the [nested-for-statement] >have completed execution, [bound-variable-expression] is Set-assigned to >the next element in [collection] in an implementation-defined manner. If >there are no more elements in [collection], then execution of the [for-each->statement] immediately completes. Otherwise, [statement-block] is >executed again, followed by [nested-for-statement] if present, and this >step is repeated.

   When the [for-each-statement] has finished executing, the value of >[bound-variable-expression] is the data value of the last element in >[collection].

Using this as a base, I think it becomes clear that a Variant assigned to a variable that then becomes the bound-variable-expression generates the "Array is locked" error in this example:

    Dim v As Variant, vv As Variantv = Array(1, 1, 1, 1)i = 1' Any of the Commented lines below generates the same RT error:For Each v In v  ' "This array is fixed or temporarily locked"'For Each vv In v    'v = 4    'ReDim Preserve v(LBound(v) To UBound(v))    If i < UBound(v) Then v(i + 1) = i + 1 ' so we can alter the entries in the array, but not the array itself    i = i + 1     Debug.Print vv,            ' 1, 2, 3, 4Next

Using 'v' as the [bound-variable-expression] creates a Let-assignment back to V that is prevented by the runtime because it is the target of an enumeration underway to support the ForEach loop itself; that is, the runtime locks the variant, thus precluding the loop from assigning a different value to the variant as would necessarily have to occur.

This also applies to the 'Redim Preserve' - resizing or changing the array, thus changing the variant's assignment, is going to violate the lock placed on the enumeration target at the loop's initialization.

With regard to Range-based assignments/iteration, note the separate semantics for non-object elements kicks in; the "external objects" provide an implementation-specific enumeration behavior. An excel Range object has a _Default property that is being called when referenced by the object name only, as in this case, which does not take an implicit lock when used as the iteration target of the ForEach (and thus does not generate the locking error, as it has different semantics than the Variant variety):

Debug.Print vbCrLf & "Case10 range in range",Set rng = Range("A1:D1") '.Cells.Cells add as many as you wantFor Each rng In rng ' (another implicit .Cells here?)    Debug.Print rng.Column,     ' 1, 2, 3, 4Next

(The _Default property can be identified by examining the Excel object library within the VBA Object Browser via highlighting the Range object ,right-clicking, and selecting "Show Hidden Members").

EDIT3: Collections

The code involving collections gets interesting and a little hairy :)

Debug.Print vbCrLf & "Case7 obj in col",Set obj = New Collection: For i = 1 To 4: obj.Add Cells(i, i): NextFor Each obj In obj    Debug.Print obj.Column,    ' 1 only ?NextDebug.Print vbCrLf & "Case8 var in col",Set v = New Collection: For i = 1 To 4: v.Add Cells(i, i): NextFor Each v In v    Debug.Print v.column,      ' nothing!Next

This is where nothing more than a genuine bug has to be considered at play. When I first ran these two samples in the VBA debugger, they ran precisely as the OP offered in the initial question. Then, after a restart of the routine following a few tests, but then restoring the code to its original form (as shown here), the latter behavior arbitrarily started matching that of the object-based predecessor above it! Only after I stopped Excel, and restarted it, did the original behavior of the latter loop (printing nothing), return. There's really no way to explain that other than a compiler bug.

EDIT4 Reproducible behavior with Variants

After noting that I'd done something within the debugger to force the variant-based iteration through a Collection to loop at least once (as it had with the Object version), I finally found a code-reproducible way of changing the behavior

Consider this original code:

Dim v As Variant, vv As VariantSet v = New Collection: For x = 1 To 4: v.Add Cells(x, x): Next x'Set vv = vFor Each v In v   Debug.Print v.ColumnNext

This is essentially the OP's original case, and the ForEach loop terminates without a single iteration. Now, uncomment the 'Set vv=v' line, and re-run: now the For Each will iterate one time. I think there's no question that we've found some very (very!) subtle bug in Variant evaluation mechanism in the VB runtime; the arbitrary setting of another 'Variant' equal to the loop variable forces an evaluation that does not take place in the For Each evaluation - and I suspect that's tied to the fact that the Collection is represented within the Variant as a Variant/Object/Collection. Adding this bogus 'set' seems to force the issue and make the loop operate as the Object-based version does.

EDIT5: A final thought about iterations and collections

This will probably be my last edit to this answer, but one thing I had to force myself to be sure I recognized during the observation of odd loop behavior when a variables was used as the 'bound-variable-expression' and the limit expression was that, particularly when it comes to 'Variants', sometimes the behavior is induced by virtue of the iteration changing the contents of the 'bound-variable-expresssion.' That is, if you have:

Dim v as VariantDim vv as VariantSet v = new Collection(): for x = 1 to 4: v.Add Cells(x,x):nextSet vv = v ' placeholder to make the loop "kinda" workfor each v in v   'do somethingNext

it is vital to remember (at least it was for me) to keep in mind that within the For Each, the 'bound-variable-expression' held in 'v' gets changed by virtue of the iteration. That is, when we start the loop, v holds a Collection, and the enumeration begins. But when that enumeration starts, the contents of v are now the product of the enumeration - in this case, a Range object (from the Cell). This behavior can be seen in the debugger, as you can observe 'v' go from Collection to Range; meaning that the next kick in the iteration returns whatever the enumeration context of the Range object would provide, not the 'Collection.'

This has been a great study and I appreciate the feedback. It's helped me understand things even better than I thought. Unless there are more comments or questions on this, I suspect this will be my last edit to the answer.