Focusing on Business Needs Leads to Highly Effective Remote Assessment Tool

Mar 23, 2023

Innovative Canadian Camera System for Remote Assessment of Trainees

Tom was excited, but the pressure was on. After weeks of learning, it all came down to this final assessment of his skills as a participant in a CNC Machinist upskilling program. Conversing with his Assessor, Tom confidently worked to complete a series of tasks.

Watching and monitoring Tom’s steps carefully was his Work Based Learning Consortium (WBLC) Assessor, but something was different. The person overseeing the test wasn’t standing beside Tom on the shop floor. Instead, his Assessor was speaking with him remotely, watching Tom’s every move via an innovative, newly developed remote assessment tool.

With potentially hundreds for CNC Machinists being trained in different plants across the country, Work-Based Learning Consortium knew that it needed to have a cost-effective telepresence assessment solution that gives assessors the ability to remotely conduct final trainee assessments. Using remote assessment rather than having to incur the cost and time of traveling to various cities, a WBLC assessor can evaluate multiple trainees on the same day, regardless of their location, without having to be physically present at different client worksites. The development of this innovative technology solution was funded by Future Skills Centre, a pan-Canadian initiative dedicated to helping Canadians gain the skills needed to thrive in a changing labour market.

No Commercially Available Solution Found

“When we first started the process of creating our remote assessment tool, I thought we’d find a commercially available solution that would meet our requirements,” says Rick Stomphorst, Employer Relations Manager at Work Based Learning Consortium. “We engaged a third-party vendor to conduct some preliminary research and make some recommendations. We looked at wearable technology and even some augmented reality devices, but as we worked through it, each had its drawbacks. We discovered that there is a gap in the marketplace. We had some very specific requirements, and, in the end, we had to design something ourselves.”

WBLC knew that some companies had policies that prevented recording on premises, but they were fine if material was live streamed, without recording. Other requirements included:

• The remote assessment solution needed to be hands-free
• It must be easily deployable and not require any training in how to set it up
• The video must be sufficiently clear and flexible for the Assessor to judge competency
• The audio must be bi-directional and functional in a noisy factory environment
• It had to be a standalone unit, capable of a run time of 1.5 to 4 hours, not require any power cables, and have its own self-contained Internet access (could not rely on access to the customer’s wireless network)
• It had to be capable of being shipped to a customer in a suitcase sized case
• Other desires: to be expandable (future proof), rigorous and safety related, and a spare unit needed to be available for 48-hour exchange support if the first unit malfunctioned

Focus on Core Business Functions

“We went through several iterations and quickly realized that we had to pull back the layers to get away from all of the whiz-bang, cool features of technology and focus on what are the core business functions that we needed to solve,” adds Stomphorst. “We also could not afford to have to spend time training people on setup or how to use the software.”

WBLC knew that there were three identifiable physical areas within a roughly 12-foot radius that the trainee would need to move around during the CNC Machinist assessment. This meant there couldn’t be one single fixed view to monitor the trainees. A camera was required that could pan back and forth between three or four different places. They needed a camera that could plan, tilt, and zoom.

“We couldn’t have a camera on a helmet because that, in itself, presented challenges. We also discovered that wearables, such as glasses, wasn’t viable. A lot of solutions weren’t safety rated and how would you wear glasses over regular glasses? We also considered security cameras but found that their resolution wasn’t sufficient enough for our needs. So we had to pivot away from those options,” says Stomphorst.

Sorting Out The Pieces

After some trial and error, WBLC chose an Insta360 Link camera that was a 4K pan, tilt and zoom webcam. It could zoom in from a distance of 8 feet and Assessors could read very granular text on the screen of a CNC machine. They had discovered that many other cameras had difficulty reading black text on a white screen and some tended to blur it out. One other benefit of the Insta360 camera was that it supports pre-set locations so that, as the trainee moved about, the camera would adjust instantly to a set viewpoint with the click of a single of a button.

They chose a Window-based tablet because Windows proved to be a general-purpose operating system that was broadly supported and a lot of products could connect with it. The tablet also came with cellular connectivity, which was crucial because they couldn’t rely on using a customer’s Wi-Fi network. During testing, they discovered that the tablet’s internal battery wouldn’t last long enough on its own, so they added an external battery that provided six hours of power.

One of the final hurdles to overcome was audio because of the noisy factory environment. The trainee had to be able to hear the assessor and the assessor had to hear the trainee. Using an onboard speaker and microphone on the tablet was ineffective in a noisy production environment. Earbuds were a problem because they may fall out, have wires, could get lost, and there would be sanitation issues when they come back. They also didn’t want cover both ears of the trainee on the shop floor for the safety factor reasons and often no-one in CNC machine shops wore full ear protection. It’s important that the trainee be able to hear people around them.

The solution was a wireless, one ear mono-cup headset with Bluetooth and a built-in microphone that is used in the trucking industry. Speaking and hearing was quite clear. It also is remarkably simple to use with one-button connectivity and the battery lasted close to 24 hours.

Mounting Components on a Rugged Frame

Everything (excluding the headset) was mounted securely on a special ruggedized frame that could be attached to a camera tripod. The tripod has reflective tape for visibility purposes on the shop floor. The unit ships essentially in three parts – it’s the frame, which houses the tablet and camera, and quickly connects to a tripod. The process to hook them together takes about two to three minutes. The third element is the headset.

“We’ve had remarkable success with our remote assessment tool. Most of the control is managed by the Assessor. All that is required on the shop floor is initial setup of the tripod and turning the unit on,” adds Stomphorst. “Reviews of our assessment tool have been very positive.”

“It’s been great,” says Pavel Wegrzyn, a monitor/coach with WBLC who conducts remote assessments and played a key role in the tool’s development. “I can (remotely) see the coordinates on the machine, what’s happening while the doors on the CNC machine are open and easily follow the trainee as they move about. We can obtain a clear view inside the CNC machine even with the camera roughly 8 feet away from the machine. It’s easy to move the view of the camera and the ability to have three or four pre-set points has been very helpful. It’s been a fun process, working with Rick and the team to develop this solution.”