Case Study: Enhancing Credly's Canvas Integration
Learn how we automate and manage digital badging within Canvas LMS, save faculty time, and enhance learners’ engagement with our efficient, integrated tools.
Impact
- Efficient workflow to issue digital badges within the Canvas LMS
- Automatically issue a digital badge for completing a Canvas Module
- Automatically issue a digital badge for completing a Canvas Assignment
- Set badge issuing criteria based on minimum passing grade
- Read More at credly.com
Project Overview
Credly’s integration with the Learning Management System (LMS), Canvas, was aimed at automating the issuance of Credly badges based on student performance metrics. In essence, the integration equipped Canvas instructors with the ability to design rules in Credly, linked with Canvas assignments/modules. Subsequently, a badge would be issued whenever a student achieved a predetermined score on the assigned task or a combination thereof.
Technology
Backend
Ruby on Rails, ActiveJobs, LTI v1, Canvas API
70%
Frontend
React, Redux
30%
Challenges Encountered
Data Synchronization
Canvas provides an API to fetch all student information but lacks an API endpoint or web hook for real-time notification of new student additions. This necessitated periodic data fetches from every page of the Canvas instances’ student API endpoint, followed by comparison with students already logged in the database, to update student data.
Module Completions
Determining module completions presented an additional challenge as we couldn’t fetch modules based on their completion status. Instead, we had to fetch modules per student, which resulted in an n+1 request issue.
Assignment Submissions
On a brighter note, we could request assignment data from a submissions endpoint, filtered by the submission time in descending order. This gave rules, hinged on assignments, superior performance, despite issuance being hindered by student existence.
Implemented Optimizations
- Asynchronous Job Handling: Each import task was divided into individual jobs that could run asynchronously, preventing process-blocking bottlenecks and allowing for incremental optimizations in future iterations.
- Intelligent Job Frequency: Canvas usage patterns were analyzed to determine data addition/update regularity, thereby limiting job frequency accordingly. Both students and student module imports, which were infrequently updated, were scheduled to run once per week to prevent server overloads.
- Batch Processing: Import jobs for both students and student modules were executed in batches to allow synchronization in separate processes, mitigate rate limiting, ensure job resumption in the event of page-specific failures, and support horizontal scaling.
- Lazy Loading of Students: In cases where Canvas's submissions API returned assignments for students yet to be imported, the system was designed to fetch and store the data for the individual student.
- Assignment Rule Creation: Instructors were encouraged to use assignments for rule creation instead of modules as a completed module typically entailed completion of all its constituent assignments.
- Job Pausing: Jobs related to dormant accounts, i.e., accounts showing no recent activity, were temporarily halted to decrease overhead.
Debugging Complexities and Solutions
In the course of our work, we encountered instances where some students who had fulfilled all rule requirements did not receive their badges as expected. The intricate nature of our system made it difficult to identify the source of these failures. For instance, while runtime errors that occurred during individual tasks could easily be traced using Rollbar, an error tracking tool, these errors generally affected the entire Canvas instance. On the other hand, failures to award badges to specific students were far more nuanced and required a more detailed examination.
In response to this challenge, a dedicated script was created to perform a series of tests on a collection of student emails. The primary question we sought to answer was: What distinguished this specific student that obstructed their badge receipt?
The script's operation was as follows:
- Presence Verification: The first test checked if the student was present in the Canvas API response. If they were, the system queued a job to synchronize students on that instance. If not, feedback was provided to the customer for resolution.
- Database Verification: If the student was already logged in the database, the API was consulted for the student's submissions and module completions.
The main objective of the script was to execute a thorough walkthrough of every Canvas API request linked to the specified students. This enabled us to determine whether the issue lay in the data output from Canvas or how we processed that input. In essence, our debugging strategy ensured that all relevant data was accurately fetched and processed, significantly simplifying the troubleshooting process.
Key Takeaways
- Asynchronous Processing: Breaking down tasks into independent, asynchronously running jobs enhances system performance and allows for better optimization.
- Usage Pattern Analysis: Understanding system usage patterns can significantly contribute to optimizing resource allocation and job frequency.
- Effective Debugging: Developing robust debugging tools can drastically simplify issue resolution in complex systems.
- Incentivizing Efficient Practices: Encouraging instructors to use assignments for rule creation over modules improved system performance.
- Proactive Management: The introduction of a mechanism to pause jobs for dormant accounts effectively reduced system overhead.
Are you looking for a seasoned professional to join your development team? Does your startup crave the expertise of a technical co-founder? Let’s get in touch!
