Data Instrumentation: The Work That Makes Analytics Possible
Dashboards tend to receive all the attention.
They are the visible part of a data system: the charts, reports, alerts, recommendations, and increasingly, the artificial intelligence applications that promise to turn organizational activity into insight. But every dashboard sits at the end of a much longer process. Before data can be stored, queried, analyzed, or visualized, something has to produce it. Someone has to decide what constitutes an event, how that event should be described, which context should accompany it, and where the resulting data should go. That work is data instrumentation.
Data does not simply appear
Most software generates some form of data. Applications produce logs. Learning management systems record transactions. Simulations update objects. Media players know when a video starts, stops, or advances. That does not necessarily mean the software produces useful data. After all, while a system log may tell us that a function executed, it may not tell us that a learner attempted a task, or that a pilot responded to a particular condition, or that a team completed an activity under a defined set of circumstances. The technical event occurred, but its operational or learning significance was never expressed.
Data instrumentation is the deliberate process of making that significance visible. It involves identifying meaningful activity within a system, capturing it at an appropriate point, and expressing it in a form that can be understood outside the system where it originated. In the learning and training domain, the Experience API (xAPI) provides a common way to describe that activity and send it to a Learning Record Store for further use.
Instrumentation is therefore different from ordinary data integration. Integration moves data between systems. Instrumentation determines what data should exist in the first place. An integration can transform an existing field, relocate a record, or combine one dataset with another, but it cannot reliably reconstruct context that was never captured.
Approaches to data instrumentation
There is no single way to instrument a technology. The right approach depends on the architecture of the system, the activity being observed, and the degree of access available to the implementation team. Several open source projects developed and supported by Yet Analytics illustrate four different instrumentation patterns.
Native application instrumentation: Unity xAPI Publisher
When developers control an application, they can instrument activity directly within its runtime. The Unity xAPI Publisher is a package designed to simplify the connection between a Unity application and a Learning Record Store. Developers can publish common lifecycle statements, define custom verbs and activities, modify statements before they are sent, and connect statement publishing to events within a Unity scenario.
This is often the most direct form of instrumentation. A developer might publish an xAPI statement when a learner enters a scenario, interacts with an object, makes a decision, completes a procedure, or exits an exercise. Because the instrumentation operates within the application, it can include details that may not be available anywhere else: scenario state, environmental conditions, the learner’s role, the object being manipulated, or the result of an action.
This approach is particularly useful in games, extended reality, and interactive simulation. Yet has used the Unity publisher as the basis for instrumentation in educational simulations where the immediate goal was not merely to record that someone opened an application, but to create a reusable framework for capturing behavioral activity across current and future Unity scenarios. The essential advantage is proximity. The instrumentation is placed where the activity actually happens.
Platform-level translation: Moodle Logstore xAPI
Sometimes a mature platform already records substantial activity through its own event or logging framework. In that case, the best approach may be to translate existing platform events into an interoperable data model.
Moodle Logstore xAPI takes events recorded by the Moodle logstore and emits corresponding xAPI statements to an LRS. Its architecture allows developers to map supported Moodle events, modify the statements produced for those events, and create mappings for additional events. This represents a different kind of instrumentation. Rather than adding tracking calls throughout the application, the implementation uses an existing event stream as its source.
Yet Analytics completed a major expansion of the plugin’s instrumentation, increasing its coverage to approximately 135 Moodle events. The supported activity includes learner, instructor, and administrative behavior across functions such as courses, calendars, forums, quizzes, individual questions, and responses.
The important point is not simply the number of events. More data is not automatically better data. Rather, the importance of the data instrumentation process means that each platform event must be interpreted. A technical log entry has to be mapped to a meaningful actor, verb, activity, result, and context. Identifiers need to remain consistent. Similar actions need to be represented in similar ways. Events that matter to an administrator may be different from those required by an instructional designer, evaluator, or organizational analyst. Platform-level instrumentation succeeds when it converts application behavior into a coherent record of activity, not when it merely exports every available log. Therefore, by following the data instrumentation process, you are improving the quality of your data.
Instrumenting activity at the edge: xAPI for VLC
Learning and performance activity does not always take place inside an LMS, browser, or centrally managed application. For example, video may be played locally. Training may be delivered in constrained environments. A media file may be distributed independently of the platform that originally assigned it. In these situations, the media player itself may be the most reliable place to observe activity.
xAPI for VLC adds xAPI instrumentation directly to VLC media player. The plugin listens for changes in playback state and produces statements containing information such as the media title, playback status, duration, current time, and progress through the video. This is a relatively compact implementation, but it demonstrates a major principle of instrumentation: instrument the point where the activity can actually be observed.
An LMS may know that it presented a link to a video. That is not the same as knowing what occurred in the media player. A course completion record may indicate that a learner reached the end of a module. It may not reveal whether the learner played the video, stopped halfway through it, returned later, or reviewed a particular portion repeatedly. But instrumenting the player closes part of that gap.
The broader lesson applies well beyond media. In distributed ecosystems, relevant activity may occur in mobile applications, simulators, physical devices, command-line tools, business systems, or equipment operating far from the central learning platform. The learning experience is not synonymous with the LMS. The instrumentation strategy should reflect that reality.
Instrumentation through observation: Federate xAPI
The most interesting instrumentation problem may arise when modifying the source systems is impractical, risky, or simply unnecessary. This is especially the case in High Level Architecture (HLA) simulation environments. Rather than requiring existing simulation federates to be rewritten, Federate xAPI joins the Runtime Infrastructure (RTI) as another federate. It can then convert selected HLA interactions and object activity into xAPI statements and send those statements to an LRS.
Its configuration determines which HLA events should trigger statements, what criteria must be satisfied, which cached object data should be retrieved, how values should populate an xAPI statement template, and where the resulting statements should be sent. This creates an observer-based instrumentation pattern. The original systems continue doing what they were designed to do. The instrumentation operates at an existing interoperability layer, where it can observe activity from across the federation and selectively express that activity as xAPI.
For established environments, this can significantly reduce the disruption associated with introducing a new data capability. It also separates instrumentation logic from simulation logic. Teams can modify filters, mappings, and statement templates without embedding every change inside the applications producing the original events. The same principle can be applied elsewhere. When a platform exposes a reliable event bus, message stream, logstore, application programming interface, or interoperability layer, instrumentation may be implemented beside the source system rather than inside it.
From telemetry to evidence
All four of these approaches begin with technical events:
A callback occurs in Unity.
Moodle writes an event to its logstore.
VLC changes playback state.
An HLA interaction arrives through an RTI.
Instrumentation turns those events into evidence. That transition requires more than software development. It requires an understanding of what the organization is trying to observe and why.
A strong instrumentation effort usually begins with questions rather than fields. What decisions will this data support? What activity provides evidence for those decisions? Which system can observe that activity? What context will analysts need later? How should identities, sessions, activities, and results be represented? What should not be collected? From there, teams can define an event model, develop or adopt an xAPI Profile, identify appropriate capture points, implement the required publishers or adapters, validate the resulting statements, and monitor the data after deployment.
This is a design-backwards approach. The desired use of the data informs the data model. The data model informs the instrumentation. The instrumentation then creates the evidence needed by the application, report, evaluation, or analysis. This is more effective than beginning at the opposite end and collecting everything a system happens to expose. That approach usually produces more volume than value.
Good instrumentation is selective
The purpose of instrumentation is not to create the largest possible dataset. Because over-instrumentation carries real costs. It increases storage and processing requirements. It makes important events harder to identify. It can create inconsistent representations of similar behavior. It may also collect personal, behavioral, or operational details that the organization has no legitimate reason to retain.
Good instrumentation captures the minimum evidence necessary to support an intended use. That requires governance. Event definitions should be documented. Identifiers and vocabularies should remain stable. Changes should be versioned. Data should be tested for both technical validity and semantic consistency. Privacy, access, retention, and security requirements should be designed into the instrumentation rather than added after data begins flowing.
The work before the dashboard
As organizations invest in analytics, adaptive systems, artificial intelligence, simulation, and workforce intelligence, data instrumentation becomes increasingly consequential. Analytical systems can only work with the evidence they receive. An AI model cannot infer with certainty that an uncaptured event occurred. A dashboard cannot supply missing context. A data warehouse cannot standardize an activity that was represented differently by every source system. Instrumentation is the work that makes those later capabilities possible.
At Yet Analytics, we have approached that work from multiple directions: publishing activity natively from Unity applications, translating Moodle platform events, instrumenting media playback in VLC, and observing distributed simulation activity through Federate xAPI. These projects differ in scale and architecture, but they share the same objective to make activity interoperable, accessible, and actionable.
So, before deciding which dashboard to build, which model to train, or which repository to purchase, organizations should ask a more fundamental question. They should ask what their existing systems are capable of saying about the activity taking place within them. Data instrumentation is the process employed to answer that question.