Category: SQL Server 2022

New Wait Stats In SQL Server 2022

Posted on by Erik Darling

Patiently Waiting

This is just a quick post to list out new wait stats in SQL Server 2022. How many will be useful, time will tell.

Some waits that I think might be interesting:

  • CXSYNC_CONSUMER (Currently in Azure)
  • CXSYNC_PORT (Currently in Azure)
  • PARALLEL_DB_SEEDING_SEMAPHORE
  • PLPGSQL

It looks like maybe automatic seeding for Availability Groups is getting the ability to process multiple databases at once, and we’re getting some parallel query waits that used to be Azure only.

I’m really scratching my head about PLPGSQL though. That’s the “programming language” that Postgres supports, which is sort of like what Oracle supports.

Time will tell!

+-----------------------------------------------------+
|                      wait_type                      |
+-----------------------------------------------------+
| ARC_IMDS_RESOURCE_INFO                              |
| BABYLON_POLICY_UPDATE                               |
| BABYLON_PULL_TASK                                   |
| BACKUP_BACKUP_MGR_MIHYBRIDINFO_RWLOCK               |
| BACKUP_LOG_IO_STALL                                 |
| BLOB_LIST_LIMIT_IO_REQUESTS                         |
| BLOB_LIST_RWLOCK                                    |
| BUFFERPOOL_SCAN                                     |
| CDC_SCHEDULERCACHE_ACCESS                           |
| CDC_THROTTLE_LOG_RATE_LOG_SIZE                      |
| CMEMDETOUR                                          |
| COLLECTOR_VIEW_LIST                                 |
| COLUMNSTORE_CSI_CACHE                               |
| COSMOSDB_INIT_MUTEX                                 |
| CXSYNC_CONSUMER                                     |
| CXSYNC_PORT                                         |
| DATA_EXPORT_COMPLETION_SYNC                         |
| DIFF_BACKUP_SELF_THROTTLING                         |
| DIRECTORY_CONTENT_LIST_CLERK                        |
| DIRTY_PAGE_THROTTLING                               |
| DISPATCHER_JOIN                                     |
| DTC_INFO_DMV                                        |
| DTC_PRECOMMIT                                       |
| DTCNEW_DWSHELLDB_PROPERTIES                         |
| EDC_DOPP_BACKGROUND                                 |
| EDC_DOPP_LOCK                                       |
| EDC_INIT_RWLOCK                                     |
| EMC_EXEC                                            |
| EMC_FABRIC_RESOLVE_SERVICE_URI                      |
| EMC_INIT_MUTEX                                      |
| EXECUTED_REQ_TABLE_STATE_RWLOCK                     |
| EXTERNAL_SCRIPT_CREATE_CERTIFICATE                  |
| EXTERNAL_SERVICE_BLOB_MESSAGE                       |
| EXTERNAL_SERVICE_CONNECTION_CLOSE                   |
| EXTGOV_ATTR_SYNC_BACKGROUND                         |
| FABRIC_ENDPOINT_SYNC_EVENT                          |
| FCB_DISKSPACE_COUNTERS_RWLOCK                       |
| FIDO_AUTOSTATISTICS_TASK                            |
| FIDO_CLIENT_STARTUP                                 |
| FOREIGN_FILE_VALIDATION                             |
| FULL_BACKUP_SELF_THROTTLING                         |
| GLOBAL_QUERY_EXTRACTOR_EXECUTE                      |
| HADR_THROTTLE_LOG_RATE_MISMATCHED_SLO               |
| HADR_THROTTLE_LOG_RATE_SLO_DOWNGRADE                |
| HADR_THROTTLE_REFRESH_MAX_SIZE                      |
| HTTP_EXTERNAL_CONNECTION                            |
| HTTP_EXTERNAL_CONNECTION_ALLOW_LIST                 |
| HTTP_EXTERNAL_CONNECTION_IPV4_BLOCK_LIST            |
| INDEX_BUILD_BUCKETIZATION_BARRIER                   |
| INDEX_BUILD_BUCKETIZATION_INFO_MAP_SYNC             |
| INDEX_BUILD_BUCKETIZATION_INFO_SYNC                 |
| LCK_REQ_TSK_PROXY                                   |
| LEDGER_BLOCK_GENERATION                             |
| LEDGER_TRUNCATION                                   |
| LOCK_SAFEMODE                                       |
| LOCK_UPDATE                                         |
| LOGPOOL_CONSUMER_DELETABLE                          |
| LSN_LOC_MAP_LOCK                                    |
| MANAGED_DISKS_CONFIGURATION                         |
| MULTITHREADED_VERSION_CLEANUP_WAIT_WORK             |
| NATIVE_SHUFFLE_OPEN_HANDLE                          |
| ORDLOCK_POPULATE_SYNC                               |
| PARALLEL_BPOOL_DEALLOCATION_WORKER                  |
| PARALLEL_DB_SEEDING_SEMAPHORE                       |
| PLPGSQL                                             |
| POLARIS_TSQL_TASK                                   |
| POPULATE_LOCK_ORDINALS                              |
| PREEMPTIVE_AAD_HTTP_EVENT_WAIT                      |
| PREEMPTIVE_COSMOSDB                                 |
| PREEMPTIVE_EMC                                      |
| PREEMPTIVE_FILE_MAPPING                             |
| PREEMPTIVE_HTTP_EXTERNAL_CONNECTION_EVENT_WAIT      |
| PREEMPTIVE_OS_GETQUEUEDCOMPLETIONSTATUS             |
| PREEMPTIVE_OS_PDH_WMI_QUERY                         |
| PREEMPTIVE_PREDICT_API                              |
| PREEMPTIVE_REPORTING                                |
| PREEMPTIVE_RG_HTTP                                  |
| PREEMPTIVE_SNI_SOCKET_BIND                          |
| PREEMPTIVE_SNI_SOCKET_LISTEN                        |
| PREEMPTIVE_SYNAPSESTREAMING_HTTP_EVENT_WAIT         |
| PREEMPTIVE_XCS_SNAPPY                               |
| PREEMPTIVE_XCS_THRIFT                               |
| PRU_PAGE_LSN_CACHE_LOCK                             |
| PVS_TRACK_PAGES_MUTEX                               |
| PWAIT_AUTO_START_AUDIT_SESSIONS                     |
| PWAIT_DBCC_FREEZEIO_MUTEX                           |
| PWAIT_DBCC_THAWIO_MUTEX                             |
| PWAIT_EXTERNAL_SERVICE_HUB                          |
| PWAIT_EXTERNAL_SERVICE_SEND_MESSAGE                 |
| PWAIT_FCS_MD_READ_AHEAD                             |
| PWAIT_FIDO_INDEXSTORE_CONNECTIONS_MANAGER_HASHTABLE |
| PWAIT_LATCH_ONLY                                    |
| PWAIT_PREEMPTIVE_OS_AUTHENTICATEDWEBCALL            |
| PWAIT_PREEMPTIVE_OS_AUTHENTICATIONTOKEN             |
| PWAIT_PREEMPTIVE_OS_CRYPTOPENSTORAGEPROVIDER        |
| PWAIT_PREEMPTIVE_OS_VSMATTESTATIONSERVICE           |
| PWAIT_PS_RBPEX_HOT_PAGES_RWLOCK                     |
| PWAIT_RBIO_IC_ACQUIRE_PAYLOAD                       |
| PWAIT_SBS_API_STATS_PUBLISH                         |
| PWAIT_SBS_IOAPI_STATS_PUBLISH                       |
| PWAIT_SYNAPSE_LINK_CAPTURE                          |
| PWAIT_SYNAPSE_LINK_COMMIT                           |
| PWAIT_SYNAPSE_LINK_DATA_EXPORT_SESSION              |
| PWAIT_SYNAPSE_LINK_DB_CLEANUP                       |
| PWAIT_SYNAPSE_LINK_END_HISTORY_SESSION              |
| PWAIT_SYNAPSE_LINK_GET_CURRENT_DB_LSN               |
| PWAIT_SYNAPSE_LINK_GET_DB_LIST,                     |
| PWAIT_SYNAPSE_LINK_GET_TABLE_HASHTABLE,             |
| PWAIT_SYNAPSE_LINK_LZN_API_CALL                     |
| PWAIT_SYNAPSE_LINK_MEM_CAP_THROTTLE                 |
| PWAIT_SYNAPSE_LINK_POPULATE_METADATA                |
| PWAIT_SYNAPSE_LINK_PUBLISH                          |
| PWAIT_SYNAPSE_LINK_UPDATE_TABLE_STATUS              |
| PWAIT_TOAD_CELL_ZONE                                |
| PWAIT_TOAD_DELTA_FORCE_ZONE                         |
| PWAIT_TOAD_OCCI_ZONE                                |
| PWAIT_TOAD_STAR_CELL_ZONE                           |
| PWAIT_TOAD_TUNING_ZONE                              |
| PWAIT_VLDB_PLANNED_FAILOVER_FORWARDER_THROTTLING    |
| PWAIT_VLDB_PLANNED_FAILOVER_START_THROTTLING        |
| PWAIT_VLDB_PLANNED_FAILOVER_STOP_THROTTLING         |
| PWAIT_XIO_REQUEST_IN_PROGRESS_LOCK                  |
| PWAIT_XLOG_POOL_EVICT_SLEEP                         |
| RBIO_AWAIT_RESPONSE                                 |
| RBIO_COMM_RETRY                                     |
| RBIO_COMM_UNINITIALIZE                              |
| RBIO_COMPLETE_LOG_READ                              |
| RBIO_CONNECTION_MGR                                 |
| RBIO_DB_RESTART                                     |
| RBIO_DB_TRANS_PRIMARY                               |
| RBIO_DBTRANSPRIMARY_SYNC                            |
| RBIO_FCB_DEFERRED_IO                                |
| RBIO_FCB_DEFERRED_IO_FN                             |
| RBIO_INITIALIZE_MUTEX                               |
| RBIO_PS_ACTOR_COLLECTION_RWLOCK                     |
| RBIO_RG_DESTAGE                                     |
| RBIO_RG_GEOREPLICA                                  |
| RBIO_RG_LOCALDESTAGE                                |
| RBIO_RG_MIGRATION_TARGET                            |
| RBIO_RG_REPLICA                                     |
| RBIO_RG_STORAGE                                     |
| RBIO_RG_STORAGE_CHECKPOINT                          |
| RBIO_UNINITIALIZE                                   |
| RBPEX_CHANGE_FILE_SIZE_MUTEX                        |
| RBPEX_CREATESNAPSHOT_RETRY                          |
| RBPEX_WRITEBEHIND_DB_STATE                          |
| RBPEXSHRINKTASK_SHUTDOWN                            |
| REPORTING_EXEC                                      |
| RG_MANAGER_VHD_GROWTH                               |
| RG_SERVER_CONFIGS                                   |
| ROW_GROUP_POPULATION                                |
| SBS_CONTEXT                                         |
| SBS_LRU_EVICTION                                    |
| SEEDING_COMPLETED_MUTEX                             |
| SEEDING_SELF_THROTTLING                             |
| SESSION_MGR                                         |
| SLEEP_RBPEXSHRINKTASK                               |
| SLEEP_SAFEMODE                                      |
| SOS_RG_MEM_TARGET_LOCK                              |
| SP_RESOLVE_DEFERRED                                 |
| SQLPAL_PREEMPTIVE_WAIT                              |
| SQP_STATS_REPORTING                                 |
| START_BACKGROUND_TASK_MUTEX                         |
| STREAMING_SERVICE_RESTART_FINISHED                  |
| STREAMING_SERVICE_SEND_KILL_PROCESS                 |
| STRIPE_META_UPDATE                                  |
| SYNAPSELINK_CAPTURE_JOBTASK_ACCESS                  |
| SYNAPSELINK_COMMIT_JOBTASK_ACCESS                   |
| SYNAPSELINK_FAILBATCH_ACCESS                        |
| SYNAPSELINK_PUBLISH_JOBTASK_ACCESS                  |
| SYNAPSELINK_SNAPSHOT_JOBTASK_ACCESS                 |
| TELEMETRY_SNAP                                      |
| THROTTLE_LOG_RATE_LOG_STORAGE                       |
| TOAD_AUTOSTATISTICS_ZONE                            |
| TOAD_DELETEBITMAP_ZONE                              |
| TOAD_DISCOVERY                                      |
| TOAD_FUTURE                                         |
| TOAD_QUEUE                                          |
| TOAD_RESOURCE_SEMAPHORE                             |
| TOAD_SHUTDOWN                                       |
| VERSION_LEASE_HASH_LOCK                             |
| VLDB_DUMP_LOG_LOCK                                  |
| VLDB_SNAPSHOT_MUTEX                                 |
| WAIT_AE_KEYADD                                      |
| WAIT_AETM_CRITICAL_SECTION                          |
| WAIT_AETM_ENCLAVE_WORKER_SLEEP                      |
| WAIT_AETM_HOST_WORKER_SLEEP                         |
| WAIT_DELTA_CACHE                                    |
| WAIT_DW_TX_EXTERNALIZATION_IO_COMPLETION            |
| WAIT_FCS_MD_RESOLVE                                 |
| WAIT_FIDO_GLMS_ASYNC_WORKER                         |
| WAIT_FIDO_GLMS_SYNC                                 |
| WAIT_FIDO_GLMS_UT                                   |
| WAIT_FOR_MS_POLL                                    |
| WAIT_FULLTEXT_CRAWL_MANAGER                         |
| WAIT_GC_IO_COMPLETION                               |
| WAIT_GLM_CONTROLLER                                 |
| WAIT_GLM_DB                                         |
| WAIT_GLM_DEK_LIST                                   |
| WAIT_GLM_SYNC_CLIENT                                |
| WAIT_GLMS_ACCESS                                    |
| WAIT_GLMS_LOG                                       |
| WAIT_GLMS_LOG_BLOCK_IO                              |
| WAIT_INDEXSTORE_COMPUTE_PARTITION_BUCKETS           |
| WAIT_INDEXSTORE_LIMIT_REQUESTS                      |
| WAIT_LCKMGR                                         |
| WAIT_LM_CREATE                                      |
| WAIT_LOG_REPLICA_MGR_HASH_LOCK                      |
| WAIT_LOG_REPLICA_ROLE_STABILITY_LOCK                |
| WAIT_LOG_REPLICA_WRITE_LEASE_PROPERTY_HASH_LOCK     |
| WAIT_ODBC                                           |
| WAIT_PHYSICAL_CATALOG                               |
| WAIT_RBAC_PROVIDER                                  |
| WAIT_RBAC_SESSION_AUDIT                             |
| WAIT_RBAC_SESSIONS_AUDIT                            |
| WAIT_RBPEX_WRITEBEHIND_CKPT_CONSISTENCY_LOCK        |
| WAIT_RETENTION_POLICY                               |
| WAIT_RM_RBIOCONNECTION_INIT                         |
| WAIT_SEQUENCE                                       |
| WAIT_SYNC_LAG_PARTNERS_LIST                         |
| WAIT_TRANSPORT                                      |
| WAIT_TSQL_TASK                                      |
| WAIT_VLF_IO_TRACKER_DRAIN_IO                        |
| WAIT_XCS_LOCATOR_FETCH                              |
| WAIT_XLOG_REPLICA_BG_TASK_LOCK                      |
| WMI_REGISTRATION                                    |
| XDB_PKG_LAUNCHER_CONNECTION                         |
| XE_LIVE_TARGET_MUTEX                                |
| XE_PREDICATE_HEAP_ALLOC                             |
| XE_PREDICATE_HEAP_FREE                              |
| XE_RING_TARGET_MUTEX                                |
| XE_SQL_TEXT_HEAP_ALLOC                              |
| XE_SQL_TEXT_HEAP_FREE                               |
| XE_SQL_TEXT_PREDICATE                               |
| XFILE_CACHE_XACT                                    |
| XFILE_DISPATCH                                      |
| XFILE_OBJECT_POOL                                   |
| XFILE_TASK_PROXY_ABORT                              |
| XLOG_BGTHREAD                                       |
| XLOG_BROKER_UNLINKING_IN_PROGRESS                   |
| XLOG_BROKER_WAIT_PAGE                               |
| XLOG_BROKER_WAITFULL                                |
| XLOG_BROKER_WAITMAXALLOWED                          |
| XLOG_EOL_REQUEST_NOTIFICATION                       |
| XLOG_GAPFILLERTHREAD                                |
| XLOG_LC_FWD_SEEDING                                 |
| XLOG_LC_REVERSE_SEEDING                             |
| XLOG_LC_SEEDING_VDL_ADVANCE                         |
| XLOG_POOL_FILLER_SLEEP                              |
| XLOG_POOL_QUERY_WAIT                                |
| XLOG_POOL_SHUTDOWN                                  |
| XLOG_SPACEMGR_INITIALIZE                            |
| XLOG_TOSFILE_GET_ITER_PROXY                         |
| XLOGREAD_SIGNAL                                     |
+-----------------------------------------------------+

Thanks for reading!

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that and need to solve performance problems quickly.

Things SQL Server vNext Should Address: How Did I Do?

Posted on by Erik Darling

Mom I Did It

A while back, I wrote a bunch of posts about things I’d like to see vNext take care of. In this post, since it’s Friday and I don’t wanna do anything, will round those up and cover whether or they made it in or not.

Well, maybe I’ll need to update the list for future releases of SQL Server 2022.

Hmpf.

Thanks for reading!

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that and need to solve performance problems quickly.

SQL Server 2022 DOP Feedback: Related Extended Events

Posted on by Erik Darling

Short Round

Sort of on the heels of yesterday’s post, here are some Extended Events related to the DOP Feedback feature new to SQL Server 2022.

Here’s the complete text of a session to collect all of the related events that I’ve noticed so far:

CREATE EVENT SESSION 
    dop_feedback 
ON SERVER 
ADD EVENT 
    sqlserver.dop_feedback_eligible_query
    (
        ACTION(sqlserver.sql_text)
    ),
ADD EVENT 
    sqlserver.dop_feedback_provided
    (
        ACTION(sqlserver.sql_text)
    ),
ADD EVENT 
    sqlserver.dop_feedback_validation
    (
        ACTION(sqlserver.sql_text)
    ),
ADD EVENT 
    sqlserver.dop_feedback_reverted
    (
        ACTION(sqlserver.sql_text)
    )
ADD TARGET package0.event_file(SET filename=N'dop_feedback')
WITH 
(
    EVENT_RETENTION_MODE = ALLOW_SINGLE_EVENT_LOSS,
    MAX_DISPATCH_LATENCY = 5 SECONDS,
    STARTUP_STATE = ON
);

There is one additional event in the debug channel called maxdop_feedback_received, but the contents of it don’t appear immediately actionable.

Defining Moments

Here are the definitions for each of the events above:

  • dop_feedback_eligible_query: Reports when a query plan becomes eligible for dop feedback
  • dop_feedback_provided: Reports DOP feedback provided data for a query
  • dop_feedback_validation: Reports when the validation occurs for the query runtime stats against baseline or previous feedback stats
  • dop_feedback_reverted: This reports when a DOP feedback is reverted

Fairly straightforward, here. Also seems like a decent set of events that you’d wanna have in place.

Thanks, Microsoft.

MAPDOP

The map values for each of these events is also available:

+-----------------------+---------+--------------------------------------+
|         name          | map_key |              map_value               |
+-----------------------+---------+--------------------------------------+
| dop_calculation_stage |       0 | SetMaxDOP                            |
| dop_calculation_stage |       1 | SetTraceflag                         |
| dop_calculation_stage |       2 | CalculateBasedOnAvailableThreads     |
| dop_calculation_stage |       3 | PostCalculate                        |
| dop_feedback_state    |       0 | NotAnalyzed                          |
| dop_feedback_state    |       1 | NotEligible                          |
| dop_feedback_state    |       2 | InAnalysis                           |
| dop_feedback_state    |       3 | NoRecommendation                     |
| dop_feedback_state    |       4 | AnalysisStoppedDueToThrottling       |
| dop_feedback_state    |       5 | AnalysisStoppedDueToMaxResetsReached |
| dop_feedback_state    |       6 | AnalysisStoppedMinimumDOP            |
| dop_feedback_state    |       7 | PendingValidationTest                |
| dop_feedback_state    |       8 | VerificationRegressed                |
| dop_feedback_state    |       9 | RegressionDueToAbort                 |
| dop_feedback_state    |      10 | Stable                               |
| dop_statement_type    |       1 | Select                               |
| dop_statement_type    |       2 | Insert                               |
| dop_statement_type    |       3 | Update                               |
| dop_statement_type    |       4 | Delete                               |
| dop_statement_type    |       5 | Merge                                |
+-----------------------+---------+--------------------------------------+

Why two are zero-based and one is not is beyond what I can explain to you, here.

Perhaps that will be addressed in a future release.

Thanks for reading!

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that and need to solve performance problems quickly.

Fixing Ordered Column Store Sorting In SQL Server 2022

Posted on by Erik Darling

Groove Is In The Heart

When Brent posted about the availability of, and disappointment with creating ordered column store indexes in SQL Server 2022, I got to work.

I can’t have my dear friend Brent being all distraught with all those fast cars around. That’s how accidents happen, and I fear he might leave the Blitz scripts to me in his will or something.

Anyway, I decided to dig in and see what was going on behind the scenes. Which of course, means query plans, and bothering people who are really good at debuggers.

Most of the problems that you’ll run into in SQL Server will come from sorting data.

Whenever I have to think about Sorts, I head to this post about all the different Sorts you might see in a query plan.

More on that later, though.

Cod Piece

In Paul’s post, he talks about using undocumented trace flag 8666 to get additional details about Sort operators.

Let’s do that. Paul is smart, though he is always completely wrong about which season it is.

DROP TABLE IF EXISTS
    dbo.Votes_CCI;

SELECT
    v.*
INTO dbo.Votes_CCI
FROM dbo.Votes AS v;

I’m using the Votes table because it’s nice and narrow and I don’t have to tinker with any string columns.

Strings in databases were a mistake, after all.

DBCC TRACEON(8666);
CREATE CLUSTERED COLUMNSTORE INDEX
    vcci
ON dbo.Votes_CCI
ORDER (Postid);
DBCC TRACEOFF(8666);

Here’s what we get back in the query plan:

SQL Server Query Plan
Tainted Sort

We’ve got a Soft Sort! What does our seasonally maladjusted friend say about those?

A “soft sort” uses only its primary memory grant and never spills. It doesn’t guarantee fully-sorted output. Each sort run using the available memory grant will be sorted. A “sort sort” represents a best effort given the resource available. This property can be used to infer that a Sort is implemented with CQScanPartitionSortNew without attaching a debugger. The meaning of the InMemory property flag shown above will be covered in part 2. It does not indicate whether a regular sort was performed in memory or not.

Well, with that attitude, it’s not surprising that there are so many overlapping buckets in the column store index. If it’s not good enough, what can you do?

Building the index with the Soft Sort here also leads to things being as bad as they were in Brent’s post.

Insert Debugging Here

Alas, there’s (almost) always a way. Microsoft keeps making these trace flag things.

There are a bunch of different ways to track them down, but figuring out the behavior of random trace flags that you may find just by enabling them isn’t easy.

One way to tie a trace flag to a behavior is to use WinDbg to step through different behaviors in action, and see if SQL Server checks to see if a trace flag is enabled when that behavior is performed.

If you catch that, you can be reasonably sure that the trace flag will have some impact on the behavior. Not all trace flags can be enabled at runtime. Some need to be enabled as startup options.

Sometimes it’s hours and hours of work to track this stuff down, and other times Paul White (b|t) already has notes on helpful ones.

The trace flag below, 2417, is present going back to SQL Server 2014, and can help with the Soft Sort issues we’re seeing when building ordered clustered column store indexes today.

Here’s another one:

DBCC TRACEON(8666, 2417);
CREATE CLUSTERED COLUMNSTORE INDEX
    vcci
ON dbo.Votes_CCI
ORDER (Postid)
WITH(MAXDOP = 1);
DBCC TRACEOFF(8666, 2417);

The MAXDOP 1 hint isn’t strictly necessary. With a parallel plan, you may see up to DOP overlapping row groups.

SQL Server Query Plan
community service

That’s why it was a popular maneuver to emulate this behavior by creating a clustered row store index, and then create a clustered column store index over it with drop existing and a MAXDOP 1 hint.

At DOP 1, you don’t see that overlap. It takes a lot longer of course — 3 minutes instead of 30 or so seconds — which is a real bummer. But without it, you could see DOP over lapping rowgroups.

If you want All The Pretty Little Rowgroups, this is what you have to do.

Anyway, the result using sp_BlitzIndex looks a lot better now:

EXEC sp_BlitzIndex
    @TableName = 'Votes_CCI';
SQL Server Query Results
capture the flag

How nice.

You can also use undocumented and unsupported trace flag 11621, which is

[A] feature flag for the ‘partition sort on column store order’ so the end result is similar, but via a different mechanism to 2417.
A partition sort is useful in general to prevent unnecessary switching between partitions. If you sort the stream by partition, you process all the rows for one before moving on to the next. A soft sort is ok there because it’s just a performance optimization. Worst case, you end up switching between partitions quite often because the sort ran out of memory, but correct results will still occur.

Chain Gang

A “reasonable” alternative to trace flags maybe to adjust the index create memory configuration option. If we set it down to the minimum value, we get a “helpful” error message:

EXEC sys.sp_configure 
    'index create memory', 
    704;

RECONFIGURE;

As promised:

Msg 8606, Level 17, State 1, Line 31

This index operation requires 123208 KB of memory per DOP.

The total requirement of 985800 KB for DOP of 8 is greater than the sp_configure value of 704 KB set for the advanced server configuration option “index create memory (KB)”.

Increase this setting or reduce DOP and rerun the query.

If you get the actual execution plan for the clustered column store index create or rebuild with the Soft Sort disabled and look at the memory grant, you get a reasonable estimate for what to set index create memory to.

Changing it does two things:

  • Avoids the very low memory grant that Soft Sorts receive, and causes the uneven row groups
  • The Soft Sort keeps the index create from going above that index create memory number

Setting index create memory for this particular index creation/rebuild to 5,561,824 gets you the nice, even row groups (at MAXDOP 1) that we saw when disabling the Soft Sort entirely.

Bottom line, here is that uneven row groups happen with column store indexes when there’s a:

  • Parallel create/rebuild
  • Low memory grant create/rebuild

If this sort of thing is particularly important to you, you could adjust index create memory to a value that allows the Soft Sort adequate memory.

But that’s a hell of a lot of work, and I hope Microsoft just fixes this in a later build.

Reality Bites

The cute thing here is that, while this syntactical functionality has been available in Azure Cloud Nonsense© for some time, no one uses that, so no one cares.

The bits for this were technically available in SQL Server 2019 as well, but I’m not telling you how to do that. It’s not supported, and bad things might happen if you use it.

I mean, bad things happen in SQL Server 2022 where it’s supported unless you use an undocumented trace flag, but… Uh. I dunno.

This trace flag seems to set things back to how things worked in the Before Times, though, which is probably how they should have stayed.

Thanks for reading!

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that and need to solve performance problems quickly.

Ordered Columnstore Indexes on SQL Server 2022 CTP 2.0

Posted on by Joe Obbish

Brent recently blogged about ordered columnstore indexes in SQL Server 2022 and had some trouble with them, so I decided to take a look into the mechanics of the feature. I’m testing on SQL Server 2022 CTP 2.0.

What does the ordered columnstore feature do?

  1. A sort operator may be added to query plans that insert into the table. The sort operator is a bit unusual in that the data may not be fully sorted.
  2. A sort operator is added when initially creating an ordered columnstore index. The level of rowgroup elimination fragmentation will depend on memory, DOP, and other factors.
  3. A sort operator is added when rebuilding an ordered columnstore index. The level of rowgroup elimination fragmentation will depend on memory, DOP, and other factors.

Technical details for insert

The sort for inserting into an ordered columnstore is a DML request sort. It appears to use the same internal mechanism as the sort that’s added for inserting into partitioned columnstore tables. The difference is that the data is sorted by the specified columns instead of a calculated partition id. In my testing, the sort appears to be a best effort sort that does not spill to tempdb. This means that if SQL Server thinks there won’t be enough memory then the data will not be fully sorted. Parallel inserts have an additional complication. Consider the following query plan image:

That is a row mode sort. It is a row mode sort because a batch mode parallel sort would put all resulting rows on a single thread which would make the parallel insert pointless. However, there’s no repartition streams operator as a child of the sort. Data is sorted on each thread in a best effort fashion. Even if there is enough memory to fully sort the data, you will end up with DOP threads of sorted data instead. The data will not be sorted globally. The split into threads will increase rowgroup elimination fragmentation.

As mentioned earlier, the sort operator does not always appear. It is not present when the cardinality estimate is very low (around 250 rows). I suspect that the same logic is used for adding the sort as adding the memory grant for compression. For very low cardinality estimates, the data will be inserted into delta rowgroups, even if there’s more than 102399 rows. By that same reasoning, I expect that there is no sort operator if the INSERT query hits a memory grant timeout.

For more information on this sort, see CQScanPartitionSortNew in Paul White’s blog post about different sort types in SQL Server.

Technical details for CREATE/REBUILD index

I spent less time looking into the sort that’s added as part of CREATE or REBUILD index. In my testing, the sort again does not spill to tempdb. The sort is also performed on a per thread basis for parallel index operations. The minimum fragmentation level will be achieved for a MAXDOP 1 operation with sufficient memory. Reducing memory or running the create index in parallel will increase fragmentation. This is unfortunate because ordered columnstore indexes do not support online index creation or rebuilds.

An unpopular opinion

I think that the community worries too much with columnstore with respect to achieving perfect segment ordering and keeping rowgroups at exactly 1048576 rows. If you perform basic maintenance and partition tables appropriately then that should be good enough for most data warehouse workloads. Most query performance issues are going to be caused by getting no elimination at all, scanning through too many soft-deleted rows, or the usual query performance problems. Scanning 11 rowgroups instead of 5 probably isn’t why your queries on columnstore indexes are slow today.

Final thoughts

As is, this feature can be described as a poor man’s partitioning. The sweet spot for this functionality feels very limited to me in its current state, but we’re still on CTP 2.0. Maybe there will be changes before RTM. Thanks for reading!

SQL Server 2022 Parameter Sensitive Plan Optimization: Does Not Care To Fix Your Local Variable Problems

Posted on by Erik Darling

–To Fix Parameter Sniffing

There are some code comments you see that really set the stage for how tuning a query is going to go.

Usually one misgiving about how SQL Server works gives way to a whole levee-breaking bevy of other ones and three days later you can’t feel your legs but dammit it’s done.

Okay, maybe it was three hours, but it felt like three days. Something about the gravitation pull of these black hole queries.

One fix I’ve been wishing for, or wish I’ve been fixing for, is a cure for local variables. I’d even be cool if Forced Parameterization was that cure, but you know…

Time will tell.

Husk

Let’s say we’ve got this stored procedure, which does something similar to the “I’m gonna fix parameter sniffing with a local variable hey why is everything around me turning to brimstone before my very eyes?” idea, but with… less of an end-of-times vibe.

CREATE OR ALTER PROCEDURE 
    dbo.IndexTuningMaster
( 
    @OwnerUserId int,
    @ParentId int, 
    @PostTypeId int 
)
AS
BEGIN
SET NOCOUNT, XACT_ABORT ON;

    /*Someone passed in bad data and we got a bad query plan,
      and we have to make sure that doesn't happen again*/
    
    DECLARE 
        @ParentIdFix int = 
            CASE 
                WHEN @ParentId < 0 
                THEN 0 
                ELSE @ParentId 
            END;
    
    SELECT TOP (1) 
        p.*
    FROM dbo.Posts AS p
    WHERE p.ParentId = @ParentIdFix
    AND   p.PostTypeId = @PostTypeId
    AND   p.OwnerUserId = @OwnerUserId
    ORDER BY 
        p.Score DESC, 
        p.Id DESC;

END;

How bad could a top 1 query be, anyway?

Fortune Teller

When we run this query like so and so:

EXEC dbo.IndexTuningMaster 
    @OwnerUserId = 22656, 
    @ParentId = 0, 
    @PostTypeId = 1;

EXEC dbo.IndexTuningMaster 
    @OwnerUserId = 22656, 
    @ParentId = 184618, 
    @PostTypeId = 2;

We come up with zip zero zilch none nada:

SQL Server Query Plan
still playing

We get a super low guess for both. obviously that guess hurts a large set of matched data far worse than a small one, but the important thing here is that both queries receive the same bad guess.

This is a direct side effect of the local variable’s poor estimate, which PSP isn’t quite yet ready to go up against.

Thanks for reading!

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that and need to solve performance problems quickly.

SQL Server 2022 Parameter Sensitive Plan Optimization: How PSP Can Help Some Queries With IF Branches

Posted on by Erik Darling

Time Served

I’ve spent a bit of time talking about how IF branches can break query performance really badly in SQL Server.

While the Parameter Sensitive Plan (PSP) optimization won’t fix every problem with this lazy coding habit, it can fix some of them in very specific circumstances, assuming:

  • The parameter is eligible for PSP
  • The parameter is present across IF branches

We’re going to use a simple one parameter example to illustrate the potential utility here.

After all, if I make these things too complicated, someone might leave a comment question.

The horror

IFTTT

Here’s the procedure we’re using. The point is to execute one branch if @Reputation parameter is equal to one, and another branch if it equals something else.

In the bad old days, both queries would get a plan optimized at compile time, and neither one would get the performance boost that you hoped for.

In the good news days that you’ll probably get to experience around 2025, things are different!

CREATE OR ALTER PROCEDURE 
    dbo.IFTTT 
(
    @Reputation int
)
AS 
BEGIN
SET NOCOUNT, XACT_ABORT ON;

SET STATISTICS XML ON;  

    IF @Reputation = 1
    BEGIN
        SELECT 
            u.Id, 
            u.DisplayName, 
            u.Reputation, 
            u.CreationDate
        FROM dbo.Users AS u
        WHERE u.Reputation = @Reputation;
    END;

    IF @Reputation > 1
    BEGIN
        SELECT 
            u.Id, 
            u.DisplayName, 
            u.Reputation, 
            u.CreationDate
        FROM dbo.Users AS u
        WHERE u.Reputation = @Reputation;
    END;

SET STATISTICS XML OFF; 

END;
GO

Johnson & Johnson

If we execute these queries back to back, each one gets a new plan:

EXEC dbo.IFTTT 
    @Reputation = 1;
GO 

EXEC dbo.IFTTT 
    @Reputation = 2;
GO
SQL Server Query Plan
psychic driving

Optimize For You

The reason why is in the resulting queries, as usual. The Reputation column has enough skew present to trigger the PSP optimization, so executions with differently-bucketed parameter values end up with different plans.

option (PLAN PER VALUE(QueryVariantID = 3, predicate_range([StackOverflow2013].[dbo].[Users].[Reputation] = @Reputation, 100.0, 1000000.0)))

option (PLAN PER VALUE(QueryVariantID = 2, predicate_range([StackOverflow2013].[dbo].[Users].[Reputation] = @Reputation, 100.0, 1000000.0)))

And of course, each plan has different compile and runtime values:

SQL Server Query Plan
care

If I were to run this demo in a compatibility level under 160, this would all look totally different.

This is one change I’m sort of interested to see the play-out on.

Thanks for reading!

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that and need to solve performance problems quickly.

SQL Server 2022 Parameter Sensitive Plan Optimization: Sometimes There’s Nothing To Fix

Posted on by Erik Darling

Best Intentions

After seeing places where the Parameter Sensitive Plan (PSP) optimization quite stubbornly refuses to kick in, it’s somewhat amusing to see it kick in where it can’t possibly have any positive impact.

Even though some parameters are responsible for filtering on columns with highly skewed data, certain other factors may be present that don’t allow for the type of plan quality issues you might run into under normal parameter sensitivity scenarios:

  • Adequate indexing
  • Row goals
  • Other filtering elements

This isn’t to say that they can always prevent problems, but they certainly tend to reduce risks much of the time.

If only everything were always ever perfect, you know?

Setup

Let’s start by examining some data in the Posts table.

First, PostTypeIds:

SQL Server Query Results
resultant

Questions and answers are the main types of Posts. The data is clearly skewed, here, and in my testing this does qualify for PSP on its own.

The thing is, there are several attributes that Questions can have that Answers can’t. One of those is a ParentId. Looking through how the top 15 or so of those counts breaks down:

SQL Server Query Results
hitherto

Okay, so! Wikis don’t have ParentIds, neither do Moderator Nominations. More importantly, Questions don’t.

The Question with the Most answers is Id 184618, with 518. A far cry from the next-nearest Post Types, and light years from the number of Questions with a ParentId of zero.

More important than loving your data is knowing your data.

To Query A Butterfly

Let’s say we have this query:

SELECT TOP (5000)
    p.Id,
    p.OwnerUserId,
    p.Score
FROM dbo.Posts AS p
WHERE p.PostTypeId = @po
AND   p.ParentId = @pa
ORDER BY 
    p.Score DESC;

The three things we care about getting done are:

  • Filtering to PostTypeId
  • Filtering to ParentId
  • Ordering by Score

Either of these indexes would be suitable for that:

CREATE INDEX 
    popa
ON dbo.Posts
(
    PostTypeId,
    ParentId,
    Score DESC
)
WITH
(
    SORT_IN_TEMPDB = ON,
    DATA_COMPRESSION = PAGE
);

CREATE INDEX 
    papo
ON dbo.Posts
(
    ParentId,
    PostTypeId,
    Score DESC
)
WITH
(
    SORT_IN_TEMPDB = ON,
    DATA_COMPRESSION = PAGE
);

With No PSP At All

Under compatibility level 150, we can run the query in a variety of ways and get nearly identical performance results:

SQL Server Query Plan
PostTypeId = 1, ParentId = 0
SQL Server Query Plan
PostTypeId = 2, ParentId = 184618

There’s a 27 millisecond difference between the two to find the first 5000 rows that match both predicates. You would have to run these in a very long loop to accumulate a meaningful overall difference.

In this case, both queries use and reuse the same execution plan. You can see that in the estimates.

With All The PSP

Switching to compat level 160, the queries are injected with the PLAN PER VALUE hint.

SELECT TOP (5000)
    p.Id,
    p.OwnerUserId,
    p.Score
FROM dbo.Posts AS p
WHERE p.PostTypeId = @po
AND   p.ParentId = @pa
ORDER BY 
    p.Score DESC 
OPTION 
(
    PLAN PER VALUE
    (
        QueryVariantID = 2, 
        predicate_range
        (
            [StackOverflow2013].[dbo].[Posts].[PostTypeId] = @po, 
            100.0, 
            10000000.0
        )
    )
)

SELECT TOP (5000)
    p.Id,
    p.OwnerUserId,
    p.Score
FROM dbo.Posts AS p
WHERE p.PostTypeId = @po
AND   p.ParentId = @pa
ORDER BY 
    p.Score DESC 
OPTION 
(
    PLAN PER VALUE
    (
        QueryVariantID = 3, 
        predicate_range
        (
            [StackOverflow2013].[dbo].[Posts].[PostTypeId] = @po, 
            100.0, 
            10000000.0
        )
    )
)

The thing is, both queries end up with identical execution times to when there was no PSP involved at all.

In other words, there is no parameter sensitivity in this scenario, despite there being skew in the column data.

Even searching for the “big” result — Questions with a ParentId of zero, finishes in <30 milliseconds.

Ah well. Gotta train the models somehow.

Thanks for reading!

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that and need to solve performance problems quickly.

SQL Server 2022 Parameter Sensitive Plan Optimization: Does PSP Work With Dynamic SQL?

Posted on by Erik Darling

No, Really

When I talk to clients about using dynamic SQL, they’re usually under the misconception that those plans can’t get reused.

That may be true under some circumstances when:

  • It’s not properly parameterized
  • You use EXEC only and not sp_executesql

Under more favorable circumstances, dynamic SQL gets run, executed, and plans cached and reused with the same frequency as stored procedures.

Now, dynamic SQL isn’t exactly the same as stored procedures. There’s a lot you can do with those that just looks a mess in dynamic SQL, especially longer bits of code.

In today’s post, we’re going to look at how the Parameter Sensitive Plan (PSP) optimization works with dynamic SQL.

Bright, Sunshiny

I just learned how to spell “sunshiny”. Don’t let anyone ever tell you there’s nothing left to learn.

To keep up the sunshiny visage of today’s post, let’s get a TL;DR here: PSP does work with parameterized dynamic SQL.

Here’s an example, using a query with a parameter eligible for the PSP optimization.

DECLARE
    @sql nvarchar(MAX) = 
        N'',
    @parameters nvarchar(MAX) = 
        N'@ParentId int';

SELECT 
    @sql += N'
SELECT
    c = COUNT_BIG(*)
FROM dbo.Posts AS p
WHERE p.ParentId = @ParentId;
';

EXEC sys.sp_executesql
    @sql,
    @parameters,
    0;

EXEC sys.sp_executesql
    @sql,
    @parameters,
    184618;

Both executions here get the option(plan per value... text at the end that indicates PSP kicked in, along with different query plans as expected.

SQL Server Query Plan
end of time

Being Dense

Writing the not-good kind of dynamic SQL, like so:

SELECT 
    @sql = N'
SELECT
    c = COUNT_BIG(*)
FROM dbo.Posts AS p
WHERE p.ParentId = ' + CONVERT(nvarchar(11), 0) + ';';

You will of course get different execution plans, but you’ll get a new execution plan for every different value that gets passed in. You will not get the PSP optimization.

This is not a good example of how you should be writing dynamic SQL. Please don’t do this, unless you have a good reason for it.

Anyway, this is good news, especially for parameterized ORM queries that currently plague many systems in crisis that I get to see every week.

Fun.

Thanks for reading!

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that and need to solve performance problems quickly.

SQL Server 2022 Parameter Sensitive Plan Optimization: A Missed Opportunity For PSP To Make A Query Go Faster

Posted on by Erik Darling

Farce

The index and proc creation script for this are a bit on the long side, so I’m going to stick them in a GitHub gist to keep the post concise, since we have some other ground to cover here.

I want to lead with the statistics object that gets used for the query, which has a single equality predicate on the parameter value to search the VoteTypeId column.

SQL Server Statistics Histogram

The relevant portion of the query is this:

FROM     dbo.Votes AS v
WHERE    v.VoteTypeId = @VoteTypeId

The histogram matches the row counts from the table precisely. Thanks, full scan!

So, what’s the problem?

Autoproblematic

To my eye, there’s sufficient skewness here to present parameter sensitivity issues.

SQL Server Query Results
skewdriver

But SQL Server disagrees!

SQL Server Extended Event
ouch dude

733 records vs. 37 million records seems appropriately skewed to me, but we get all of the typical parameter sensitivity symptoms.

Plansplosion

Let’s get ready to rumble, etc.

EXEC dbo.VoteSniffing 
    @VoteTypeId = 4;

EXEC dbo.VoteSniffing 
    @VoteTypeId = 2;

Here are the plans:

SQL Server Query Plan
dirty thirty

Take a moment here to admire the somewhat confusing plan timing that Batch Mode operations are presenting here: The bottom query runs for around 32 seconds.

Up to the Nested Loops join, we use ~17 seconds of wall clock time with our serial execution plan. The Hash Match operator runs for 15 seconds on its own, in Batch Mode.

Parameter sensitivity still happens in SQL Server 2022.

Reversi

If we run the procedure in reverse order, using 4 to cache the plan and then 2 to reuse, we get a similar regression:

SQL Server Query Plan
little things

Now, okay, let’s pause for a minute here. I need to backtrack, but I don’t want to spend a ton of blogspace on it. I’m also going to put the information in a very snooty block quote.

If we re-run the procedure from the section up above to search for VoteTypeId 2 a second time, Memory Grant Feedback will fix the spill at the Hash Join, and bring the total execution time down to about 15 seconds.

That is an improvement, but… Look at the plan here. If VoteTypeId 2 uses a plan more suited to the number of rows it has to process, the overall time is around 4 seconds, with no need for a memory grant correction.

The second plan for this execution sequence, searching for VoteTypeId 4 second in order, the overall time goes from 0 seconds and 0 milliseconds to 1.4 seconds. The big plan does not make searching for infrequent values faster.

So you see, the big plan isn’t always better.

Missed Connections

Hey, look, this is the first CTP. Maybe stuff like this is still getting ironed out. Maybe this blog post will change the course of history.

I am only moderately influential in the eyes of Microsoft, though, so perhaps not.

Anyway, this seems like a situation with sufficient skew to produce the Dispatcher plan and then additional sub-plans to account for far different row counts present in the VoteTypeId column.

If this scenario (and other similar scenarios) is outside the scope of the feature, query tuning folks are still going to have a whole lot of parameter sensitivity issues to deal with.

And should that be the case, I’d like to speak to the manager.

Thanks for reading!

Going Further

If this is the kind of SQL Server stuff you love learning about, you’ll love my training. I’m offering a 75% discount to my blog readers if you click from here. I’m also available for consulting if you just don’t have time for that and need to solve performance problems quickly.