The Combined Power of RISC & Yet Analytics
People post questions and answers every day on public forums like GitHub or on-line ‘enthusiast forums’ of every variety as well as closed systems like the learning management system within a company or school. What if we could analyze users’ interactions with a technical question and answer forum to identify topics that may warrant additional training or performance support materials, identify subject matter experts and users that may need an individual learning intervention to help them succeed?
To answer these questions, we connected two xAPI enabled systems for a quick and easy end-to-end solution: the RISC Virtual Training Assistant LMS and the Yet Analytics xAPI Learning Record Store.
The RISC Virtual Training Assistant LMS is the first fully xAPI enabled LMS – meaning that any interaction a learner has within the LMS is tracked and sent as an xAPI statement to a connected Learning Record Store (LRS). Everything from enrolling in a course to launching content is automatically recorded, and this level of data resolution carries across into the RISC LMS forums.The Yet Analytics xAPI LRS is the first ADL certified LRS on the market and specializes in data visualizations. Yet’s out-of-the-box analytics tools help users identify trends as interactions build within the LRS. These data visualizations operate in real time and can be used to see the impact of learning investment, evaluate the effectiveness of learning resources, and define the enterprise experience graph.
The beauty of connecting two xAPI enabled systems is that setup is a breeze. No middleware, no crazy technical expertise is needed – just an endpoint and credentials. Implementation could be done easily by copying and pasting credentials to the LMS. One of the goals of xAPI is portability of records and the Yet Analytics LRS makes configuration easy.
Exploring Forum Interaction Data
We created a series of Forums in different topic areas and made them live for students to post questions and receive responses. This was done using the native forum functionality in the RISC VTA LMS which allows for forum responses to be voted up or down and for the person posting a question to mark a response as the “Accepted Answer”.
Hot Topics & Influencers
As soon as post are made to the forums, a web of relationships began to grown in the Yet Analytics data visualizations. Links show how students (actors) interact with the different forums (objects) the linkage represents the ‘verb’ defining the interaction. You can see in the screenshot below where some interactions read “joined” and others read “replied”. As a student interacts with more forums, the circle representing the student grows, as a particular forum is interacted with it grows making it easy to see at a glance what are the hot topics and who are the users with the most interactions. The linkage lines will also thicken as interactions between the actor and the object build.
Clicking on an actor within the visualization lets you drill down into more detail about that actor specifically and their interactions.
Conversely, clicking on an object (the forum title in this case) makes the object the focus of the network report. It will then show all the people that interacted with that specific object.
Time Sinks, Helpers, and Help Requesters
While the network charts provide a great overview of interactions, the Yet Analytics platform also lets you explore trends by different variables. In the image below, we can see each student that interacted with the forum and the number of times they voted up, voted down and replied.
Simply changing the field on which to group changes the chart to show – for example, top objects (those interacted with the most) by verb.
Or even top verbs by actor. In a matter of minutes, it is easy to see where your users are spending their time trying to sort answers or helping guide those with lower technical skills. Of course you can see other verbs (note that Launched is disabled below).
Of course if we need to drill down into the specific statement data proper, we have access to do so through a raw statement view.
Exploring the high resolution RISC forum data with the Yet LRS visualizations, we can quickly and easily identify where time is getting spent, what the hot topics are, and who the highly engaged forum influencers are.
This process worked without any manipulation of data because RISC is very sensitive to sending ‘good’ xAPI statements and the Yet LRS has been designed to validate statements being written to the LRS and reject those that are not valid.
One thing to be mindful of as you establish your xAPI data systems is consistent data entry. To learn more, review articles like the xAPI Quarterly’s Expanding the Vocabulary of xAPI written by Jason Haag. Many tools use a generic “experienced” verb for all interactions. Students experience pages in a course, students experience a test question or experience a forum. This definition isn’t granular enough to help us assess value. If you can split those “experiences” into something more granular like “replied”, “voted up”, “voted down” or “accepted” as you can see in the screenshot above, the analytics are a great deal more meaningful. I can see with little analysis who my star performers are, who is lagging and what topic areas are receiving the greatest focus.
Getting Started with xAPI
One of the biggest pieces of advice we can offer for organizations looking to get started with xAPI is to make a plan going in for what you want to measure – this will help you make sure you’re getting the data fidelity you’ll need to answer real learning and business questions. Established and trusted xAPI experts like the RISC and Yet Analytics teams are happy to help you design the best xAPI solutions for your needs.
Founded in 1992, RISC, Inc. is a pioneer company in Learning Management software and an early adopter of xAPI. The company’s Virtual Training Assistant suite is the culmination of more than 25 years of continuous enhancement driven by customer and industry-solicited input. RISC software users represent leaders in a broad range of industry and government entities from Koch Industries, the City of Berkeley, Food Lion, Hach, Syniverse, and ASRC Energy. Visit http://risc-inc.com. Follow @RISC_Inc.
About Yet Analytics:
Bringing new technologies to market in the human capital space, Yet Analytics builds products that revolutionize learning and talent development. Yet’s suite of technologies includes the world’s most visual learning record store, used by organizations to measure the health of their learning and development initiatives. Customers use Yet’s technology to see the impact of learning investment, evaluate the effectiveness of learning resources, and define the enterprise experience graph.
Interested in implementing xAPI in your organization?