[3.0] Perf: Skip visiting table inside ColumnExpression

[smitpatel]

Description

An algorithmic issue in the 3.0 query code can cause execution time to grow quadratically with increasing number of includes and entity properties. This can make queries both compile and execute much slower in some scenarios.

Customer Impact

Some queries with many Include calls could be so slow that they appear to hang.

How found

Customer reported.

Test coverage

We don't currently have perf testing for queries of this level of complexity. Perf testing is something we are already actively working on.

See below for local perf numbers before and after the change--in this case we're going from 175 seconds to 19 milliseconds.

Regression?

Yes, regression in perf from 2.2.

Risk

We have spent some time deciding what to do now and going forward. For now, we have a low-risk fix that is not 100% architecturally correct, but which fixes the issue in a safe way for the current code. (The current code only uses SelectExpression in certain ways, for which this fix is correct.) We are planning to follow up in 3.1 with a more architecturally sound solution.

---

Resolves #17455

_query = _context.Blogs
                .AsNoTracking()
                .Include(x => x.Posts)
                .ThenInclude(x => x.PostInstances)
                .ThenInclude(x => x.Comments)
                .ThenInclude(x => x.AcquiredComments)
                .ThenInclude(x => x.Tag_AcquiredComments)
                .ThenInclude(x => x.Tag);

5 levels of collection include and 1 reference include. Each class containing 5 properties (+ 1 for shadow prop as needed for FK).
No data on server side. (no shaper code executed)
Run includes time to compile the query and execute it against the server.
On preview9 package

|   Method |    Mean |   Error |  StdDev |
|--------- |--------:|--------:|--------:|
| RunQuery | 174.9 s | 1.927 s | 1.803 s |

After the fix

|   Method |     Mean |     Error |    StdDev |
|--------- |---------:|----------:|----------:|
| RunQuery | 19.17 ms | 0.1993 ms | 0.1864 ms |

[smitpatel]

cc: @roji

[AndriySvyryd]

File an issue to revisit this (pun intended)

[smitpatel]

@AndriySvyryd - #17337

[ajcvickers]

@smitpatel I assume you saw this is approved and can now be merged?