Skip to content

[receiver/sqlserver] Add Always On Availability Group database replica metrics#49476

Draft
splunk-shanu wants to merge 11 commits into
open-telemetry:mainfrom
splunk-shanu:sqlserver-ag-metrics
Draft

[receiver/sqlserver] Add Always On Availability Group database replica metrics#49476
splunk-shanu wants to merge 11 commits into
open-telemetry:mainfrom
splunk-shanu:sqlserver-ag-metrics

Conversation

@splunk-shanu

@splunk-shanu splunk-shanu commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Adds 6 new opt-in metrics for monitoring SQL Server Always On Availability Groups, sourced directly from sys.dm_hadr_database_replica_states:

  • sqlserver.availability_group.database_replica.estimated_data_loss — estimated potential data loss if failover occurred now, expressed as age in seconds of the oldest unsent log record on the primary replica
  • sqlserver.availability_group.database_replica.estimated_recovery_time — estimated time in seconds to redo all log records in the redo queue on the secondary database replica
  • sqlserver.availability_group.database_replica.hardened_latency — time in seconds between a log record being generated on the primary and hardened to disk on the secondary (SQL Server 2016+)
  • sqlserver.availability_group.database_replica.queue.size — log send and redo queue backlog in bytes, distinguished via sqlserver.availability_group.queue.type attribute (log_send, redo)
  • sqlserver.availability_group.database_replica.log_send.rate — rate at which log data is being sent from the primary database replica to the secondary (bytes/s)
  • sqlserver.availability_group.database_replica.redo.rate — rate at which log data is being redone on the secondary database replica (bytes/s)

All metrics are scoped per database per replica and include sqlserver.availability_group.name, db.namespace, and sqlserver.replica.name attributes. All metrics are disabled by default and require a direct connection to SQL Server with Always On Availability Groups enabled.


Link to tracking issue

(fill in if applicable)


Testing

Unit tests added covering metric recording from sys.dm_hadr_database_replica_states. Test data files added under testdata/.


Documentation

documentation.md regenerated via make generate to include the 6 new metrics with their attributes, units, and extended documentation notes.

Authorship

  • I, a human, wrote this pull request description myself.

Comment thread .chloggen/sqlserver-ag-metrics.yaml Outdated
@akgrover

akgrover commented Jul 6, 2026

Copy link
Copy Markdown

@splunk-shanu do these metrics require additional permissions to what's documented right now?

@thompson-tomo thompson-tomo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous attempt in #48377

Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
gauge:
value_type: double
attributes: []
sqlserver.availability_group.estimated_data_loss:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming doesn't follow semconv naming guidance and is ambiguous.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

follows {system}.{area}.{property} pattern correctly

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't satisfy

Be precise. Attribute, event, metric, and other names should be descriptive and unambiguous.

For instance why is estimated in the name? What data is being lost?

At the same time, name gives the impression that it is the data has been lost but the measurement is in seconds. Data is not measured in seconds but rather bytes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated to more descriptive description, but name will have estimate as it is defined in dmv please refer dmv for more detail.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My quote is taken directly from the naming document https://opentelemetry.io/docs/specs/semconv/general/naming/

The name of the metric doesn't need to match db column name.

value_type: double
attributes: [availability_group.name, db.namespace, replica.name]
extended_documentation: This metric is only available when the receiver is configured to directly connect to SQL Server and the instance has Always On Availability Groups enabled.
sqlserver.availability_group.estimated_recovery_time:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming doesn't follow semconv naming guidance.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

follows {system}.{area}.{property} pattern correctly

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't satisfy

Be precise. Attribute, event, metric, and other names should be descriptive and unambiguous.

For instance why is estimated in the name?

At the same time it is a property of the replica rather than the group.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated to more descriptive description, but name will have estimate as it is defined in dmv please refer dmv for more detail.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My quote is taken directly from the naming document https://opentelemetry.io/docs/specs/semconv/general/naming/

The name of the metric doesn't need to match db column name.

value_type: double
attributes: [availability_group.name, db.namespace, replica.name]
extended_documentation: This metric is only available when the receiver is configured to directly connect to SQL Server and the instance has Always On Availability Groups enabled.
sqlserver.availability_group.hardened_latency:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming doesn't follow semconv naming guidance and is ambiguous.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

follows {system}.{area}.{property} pattern correctly

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardened latency is not a property of the group but rather a replica.

extended_documentation: This metric is only available when the receiver is configured to directly connect to SQL Server and the instance has Always On Availability Groups enabled.
sqlserver.availability_group.log_send.queue.size:
enabled: false
description: Amount of log records of the primary database that has not been sent to the secondary replicas.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Description contradicts unit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

extended_documentation: This metric is only available when the receiver is configured to directly connect to SQL Server and the instance has Always On Availability Groups enabled.
sqlserver.availability_group.log_send.rate:
enabled: false
description: Rate at which log records are being sent from the primary database replica to the secondary replica.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Description contradicts unit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

extended_documentation: This metric is only available when the receiver is configured to directly connect to SQL Server and the instance has Always On Availability Groups enabled.
sqlserver.availability_group.redo.queue.size:
enabled: false
description: Amount of log records of the secondary database that has not yet been redone.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Description contradicts unit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

extended_documentation: This metric is only available when the receiver is configured to directly connect to SQL Server and the instance has Always On Availability Groups enabled.
sqlserver.availability_group.redo.rate:
enabled: false
description: Rate at which log records are being redone on the secondary database replica.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Description contradicts unit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
Comment thread receiver/sqlserverreceiver/metadata.yaml Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants